twcbackup Guide

Overview

twcbackup is a backup software which is usually  configured to run once every night. During this nightly backup run, it automatically determines which files have been changed, and adds these to the backup pool. Within this document, only the case where the backup pool is managed by bfsync is described. Other (non-bfsync) backup methods are available (like a tar-based backup method), but the recommended method is to use only the bfsync backup method.

In this case, all data on the backup media is checksummed using SHA1 hashes, and every SHA1 hash is only added once to the backup. This also means that if you move files, the next backup won't contain the same files again, as the SHA1 hash is already known. So TwcBackup combined with bfsync provides file-level deduplication, and if the backup contains data from more than one machine, many system files (from /usr/lib or /usr/bin) will only be stored once on the backup media.

The backups will not only store files, but also symlinks, hardlinks, devices and other special file, as well as user/group and permissions. This means that the backup will contain all the information required for restoring all files of a system completely.

Installation

twcbackup follows the usual

# ./configure
# make
# make install

installation procedure. Note that if you use bfsync backup methods (as recommended in this guide), you need to compile and install bfsync first (which also is done using ./configure ; make ; make install).

Configuration

twcbackup needs the system wide configuration file /etc/twcbackup. This file can be created or edited by starting

# twcbackupconf

as root. need to be filled out in any case.

As an example, we assume that your system contains two filesystems, one mounted as / and the other mounted as /home, and you want to backup both. This is configured in four steps:

1. Configure Destination and Report

The Destination and Report pages need to be filled out for every configuration. Since twcbackupconf has its own context sensitive help, we don't describe the details here, as they are in the context sensitive help. Use the "Help" button to get context sensistive help in the configuration program.

2. Add a BFSync Repository

Go to the BFSync Repository entry on the left side, and use the Add button on the right side to add the repository. Each repository will have its own file pool of SHA1 hashes and a history that maps file names/permissions/... to these hashes. So deduplication only takes place between files stored in the same repository.

Each field will be checked by the configuration program every time you edit it, and fields that are not valid are marked with *. For the repostitory, a suitable Name could be main, and a suitable Database Cache Size could be 256 Mb (this depends on the total number of files that are to be backuped - see context sensitive help). The other settings mainly affect when old backups can be deleted, and are also documented in the context sensitive help.

3. Add a BFSync Backup

Go to the BFSync Backup entry on the left side, and use the Add button on the right side to add the repository.

Each BFSync backup needs to have a unique name. In this case the "Name" could be root_home (as it contains the filesystems /=root and /home). One of the repositories defined before should be used as Repo, in this case main. Finally "Include Dirs" should be set to / and /home. The "Only backup Files on the same Filesystem" property should be checked, otherwise twcbackup would recurse into filesystems like /proc which don't want to backup.

4. Save your TwcBackup configuration

If you have finished the configuration steps listed above, you should save the configuration to /etc/twcbackup. This is the default configuration filename for TwcBackup.

Checking your configuration

TwcBackup allows checking if a configuration file is valid, without actually performing the backup. This can be done as root, using

# twcbackup --check

Running twcbackup every night (via cron)

To run twcbackup automatically every night, add this line to /etc/crontab:

0 3     * * *   root    /usr/bin/twcbackup -q

This will run twcbackup at 3:00 in the morning. Since the backup log is sent by email, it's also useful to use the -q option to suppress printing the backup log after the backup is done.

External Disks

TwcBackup can either be configured to backup to a local directory (which simply requires a valid path), or to external disks. External disks need to be labelled before using them with a consistent filesystem label. One way of setting it is using e2label for the ext2 family of disks.

Once labeled, the UUID of all disks need to be determined and then each disk needs to be added as Media using twcbackupconf. The UUIDs can for instance be figured out by running:

# blkid