bfsync

About Bfsync

Bfsync is a file-synchronization tool which allows to keep a collection of big files synchronized on many machines. To do this, bfsync maintains a global and local history of changes; every time the file collection is changed on one machine, an entry in the local history is made. Bfsync allows to automatically merge this local history with the global history where possible, and offers manual conflict resolution in cases where this is not possible.

Due to history synchronization, each bfsync checkout knows precisely which files are part of the file collection. Therefore, it can determine which file contents (data blobs with SHA1 contents) are present in a checked out repo, and which are missing. The user can transfer file contents between repos using bfsync get/put, so that after transfer, the checkouts will be complete (containing both: the history and the file contents required).

To sum it up, bfsync behaves not unlike version control systems like git or svn, however it behaves reasonable when the file collection is big (like houndreds of gigabytes).

The main interface to bfsync is a FuSE filesystem, so it is possible to manage your data with a file manager or copy new data into the repository using rsync. As soon as you "commit" the changes, they are entered into the local history and if you "push/pull" the changes, they become part of the global history. There is no need to transfer all new data to a central server during "push/pull". Only the history needs to be transferred, the contents of the files can be exchanged between different machines without need for a central server (although you can have a central server containing all data if its practical for you).

Bfsync is implemented in C++ and Python and licensed under the GNU GPL version 3.

Bfsync & Backup

Bfsync was designed from the start to store all information a conventional filesystem would store. So it stores the user/group, permissions, symlinks, hardlinks, block devices, ... so it is possible to do a full system backup into a bfsync repository. By using rsync and then bfsync commit every day, a backup which automatically deduplicates at file level can be implemented. See the BACKUP WALKTHROUGH section of the bfsync manpage for details.

What Bfsync can do

What Bfsync can't do

Getting bfsync

The current version of bfsync is bfsync-0.3.7, and can be downloaded here:

The git repository can be cloned like this:

git clone http://space.twc.de/public/git/bfsync.git

and browsed via gitweb.

Overview of Changes in bfsync-0.3.7

[ more... ]

Building bfsync 0.3.7 on Ubuntu

The following packages are required (and need to be installed with apt-get install before building):

If you want to build bfsync from git, you also need:

Documentation (bfsync-0.3.7)

Issue tracker

If you have issues/bugs related to bfsync, you can use the issue tracker on github.

Discussion group

There is a group on google groups you can join to discuss questions related to using bfsync and development topics.

Older versions

Older versions are

Valid HTML 4.01 Transitional