April 2015

You are currently browsing the monthly archive for April 2015.

VPS hosting is available, affordable, and just a little bit scary.  In order to alleviate some of risk that is taken when moving to a VPS that is not under our benevolent control, we need to set up a reliable data backup solution.  The setup that I incorporated involves a server in the cloud backing up to my box.com account.  To accomplish this on CentOS 6, you can just use the following commands in a script that is executed periodically:


echo “Files backup to cloud”
eval $(gpg-agent –daemon)
export PASSPHRASE=””
export FTP_PASSWORD=”your password”
echo “Files backup to cloud”
duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY –full-if-older-than 4M /var/spool/duplicity/ webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-but-n-full 4 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-inc-of-but-n-full 2 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/duplicity
echo “Database backup to cloud”
duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY –full-if-older-than 4M /var/spool/holland/ webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-but-n-full 4 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland && duplicity –use-agent –encrypt-key YOUR_ENCRYPTION_KEY remove-all-inc-of-but-n-full 2 –force webdavs://YOUR_BOX_COM_ACCOUNT@dav.box.com/dav/lynetSky/holland


These commands will instruct duplicity to make a full backup every four months (–full-if-older-than 4M).  When this script runs in the in-between times (say 2 months after the last full backup), it will create an incremental backup.  Duplicity will keep four full backups.  This is specified by the remove-all-but-n-full 4 directive in the command.  Specifying remove-all-inc-of-but-n-full 1 tells duplicity to remove all incremental updates except in the case of the last two backup set.  A backup set includes the last full backup and its incremental backups.  Yes, this is a bit complicated.  Yes, it is worth it.

The end result is this: in sixteen months, there will be four full backups (a new full backup is created every four months).  The newest two of these will have the incremental backups as well as the full backups.  This way, file history will be completely preserved for the most recent eight month period.  If we are really desperate, we can recover a file that twelve months old or sixteen months old.

We need to consider what happens at month seventeen.  We will have four full backups, the oldest full backup will be seventeen months old.  We should not expect to be able to recover files that are over sixteen months old.  When we continue on to our twentieth month  the oldest full backup will be removed and we will be back to only having sixteen months of recoverable data.  Always think of your maximum time to recover a file as (–full-if-older-than 4M * remove-all-but-n-full 4 == 4 months * 4 full backups = 16 months of full backups).

Storing four full copies of the data on a backup is (most likely) overkill for what I am doing.  These options were added when I decided that a full backup on another server lacked one thing: files that were deleted on the server were never deleted in the backup location.  Then, of course, my mind wondered into using rsync instead of duplicity.  It’s true, if all we need is a single backup of the files, rsync can provide a better solution (I do this with my personal photos and videos at home).  However, the point in time snapshot duplicity provides can be used for forensics and change tracking.  This solution as scripted above isn’t one size fits all.  Use the tools and settings that work the best for you.

This setup absolutely saved this site and the other sites I run.  The SAN that this VPS (at the time of this writing) is running on became corrupted and all of the data would’ve been lost had I not used an external site for backups.  If you are someone who is worried about the security of your backups, remember that duplicity has an automatic encryption algorithm built in.  Security of your data is there, just make sure to backup your keys so you can read the files.

To conclude: backups can save you important information; the cloud is a great place to store your backups; duplicity is a great tool that can automate this process.

Tags: , ,