Site Overlay

Efficient backup of your Raspberry Pi

By Jens Libuda

Anyone who runs critical services such as web servers or private clouds on a Raspberry Pi is well advised to produce regular backups. The usual procedure is to remove the SD card, put it into your notebook and produce a card image with dd. This approach is as popular as it is inefficient: removing the SD card means that the system has to shut down completely. It is therefore impossible to run these backups unattended at regular intervals. Furthermore, dd also backs up empty spaces on the SD card, i.e. time and disk space are wasted. The raspiBackup script promises to be a more intelligent solution, allowing “hot backups” onto local and remote devices while the system is running.

1. Remote backup destination should be NFS

Although raspiBackup allows for mounting local media, we would prefer to perform the backup of our Pi over the network. Keep in mind, that our Pi ZeroW does not even come with a grown up USB port to attach a memory stick to.

The protocol used for sharing data over the network should satisfy the following basic requirements:

  • it should allow symlinks and so called “hard links” in order to work efficiently with rsync which acts inside of raspiBackup
  • it should follow the logic of chmod and chown permissions for files and directories on Linux systems

Besides being complicated to set up on the file server, the popular samba protocol fails these requirements. The same goes for the handy sshfs protocol. Neither of these protocols can handle hard links. We therefore go for the NFS protocol which should suffice our requirements.

1.1 Prepare NFS directory on the destination machine

There is a very good description on sharing files over the NFS system on howtoforge.com. Make sure that the nfs-kernel-server package is installed on the destination machine.

We first create the directory which is to be shared. We deviate from the instruction to create such a directory under /var/nfs because we do not want to have write-intensive operations on the SSD and our /var partition on the server would have insufficient disk space to hold large amounts of backup data from the Raspberry.

Next we will have to edit /etc/exports. As the raspiBackup script will perform chown operations in the destination folder, the NFS access must be granted in a way that super user operations can be performed. This is done by the switch no_root_squash in the config file.

Note that each client to which the file system needs a line. So if you have another client with host name pi3bp, you have to add another entry to /etc/exports. Also note, that you can only use the host name if you have associated it to an ip-address in /etc/hosts. If this is not the case, you have to use the ip-address, e.g.:

We refresh the current NFS configuration and start the NFS service:

1.2 Prepare the Raspberry client to access the NFS drive

Make sure you have the nfs-common package on your Raspberry.

Next we need a point in the Raspberry directory structure to mount the remote NFS on. We therefore create a directory:

As we want to access this directory through raspiBackup with super user privileges, there is no need to alter ownership, group membership and access privileges.

Mount the remote file system to that directory on the raspberry by:

We are now connected to the remote file system on the server which stores the backup.

2. Download, Installation and Configuration of raspiBackup

I tried my luck with the raspiBackup script. Download links and installation instructions can be found at the raspiBackup home page.

2.1 Initial Setup

After downloading the raspiBackup configuration and programm code resides under /usr/local/bin. For configuration and updates, call

2.2 Amendments to raspiBackup.conf

Backup path

We still need to edit /usr/local/etc/raspiBackup.conf and amend our mounted NFS directory as the destination:

Exclusion of directories

While raspiBackup excludes typical directories that are “not worth” being backed up (e.g. /tmp) automatically, it may make sense to exclude further directories.

A typical example is the local Nextcloud/ownCloud directory, which is already remotely stored on the Nextcloud server. As that server is also backed up, including the local /home/pi/ownCloud directory would mean duplicate backups.

These exclusions can be defined by the DEFAULT_EXCLUDE_LIST parameter. Entries in that list must follow rsync conventions, i.e.

  • each exclusion must be introduced by --exclude=<relative path>,
  • unlike when specifying source and destination paths the “trailing slash” has no effect. A directory in the exclude list will be excluded completely – with or without trailing slash.
  • each directory must have a complete serparate exclude entry in the list. If you want to exclude dir1 and dir2, then the entrys must not be--exclude=dir1 dir2 but --exclude=dir1 --exclude=dir2

The below is an example path which excludes the ownCloud directory and the directory holding the media for the minidlna server:

SUSPENDING services DURING BACKUP

In order to prevent inconsistent data, it is advisable to stop services like mysqld, nginx, cron etc. After the backup is completed, each service must be restarted again. This is done by the DEFAULT_STOP_SERVICES and DEFAULT_START_SERVICES parameters.

Keep in mind that the order in which the services are started or stopped can make a difference. Below is an example of what the services entries might look like:

3. Performing Backups

Keep in mind that backups over the network can be extremely time consuming. The following table summarizes the time needed for backing up different Raspbian systems over NFS. “Initial backup” implies that each and every file on the source system has to be transferred over the network. After that backup is completed, another backup is relaunched (“resync”). As there have been no modifications to the system, it can be assumed that this is an (almost) pure check if each file on the source system has an identical counterpart on the backup system.

The time needed for a full sync can be interpreted as the “upper bound” of the backup time needed for a system of the given size while the “resync” time is a proxy for the lower bound.

[table id=20 /]

3.1 Manual start of the backup

Manual start of a backup is triggered by:

4. Restore

Restore instructions are based on the procedures laid out on the raspiBackup page. We assume that the backup snapshots are on an NFS exported file system on a remote machine that we call the backup server.

Unlike the backup procedure, the restore procedure is not a “hot restore”, i.e. we have to remove the SD card from the Raspberry and put it into an Ubuntu machine that has raspiBackup installed and that we call the restore machine.

  1. Put the SD card  tho which the backup shall be restored into a Ubuntu machine that has a SD card reader and raspiBackup installed.
  2. Export the backup directory on the remote server via NFS (see section 1.1, make sure your ubuntu restore client holding the SD card is listed in the  /etc/exports entries on the backup server).
  3. On the restore machine, mount the NFS directory
  4. On the restore machine, determine the path to the SD card. You can find that path by running sudo parted -l. In our case, it is /dev/mmcblk0.
  5. Once you know the default path, start the backup procedure, you need to be root:

As can be seen from the output, it takes about 1/2 hour to restore a 3 GB test system.

Unmount the SD card, put it into your Raspberry Pi and reboot the restored system.

5. Trouble shooting

I have had an issue with ACL (access control list) attributes for the /media/pi directory. The raspiBackup FAQ recommends removing the acl entry from the /etc/mke2fs.conf file. It turns out this modification has not helped at all. Whether the acl entry is removed on the Raspberry, on the server or on both systems – raspiSync will always abort the backup process saying it cannot handle ACL entries.

The solution was therefore to remove the ACL entry directly from the /media/pi directory. ACL flagged directories can be recognized by the + in the ls -alh output (e.g. drwxrwxr-x+).