YARB

Jonathan Vaught is the new maintainer of the backup_fu plugin that makes Amazon S3 backups “redonkulous” (I crack myself up sometimes).

You can checkout its new home over on GitHub at:

http://github.com/gravelpup/backup_fu

thanks, Jonathan!

Nothing’s ever safe and stable on the ‘net. You can have the most expensive servers money can buy, the best broadband service on the ‘net, and the most high-end computer in the market but your system is still prone to crashes. That’s why a good knowledge on making backups is a must.

Just a tip, because I’ve finally bumped into an issue with this on one of my servers.

Backup_fu first dumps files to RAILS_ROOT/tmp/backup. This is nice and all to have a local copy of the backup … but it never does any cleaning out.

If you are looking for a solution that handles cleaning up old backups, I believe you can Google around and there are some out there like that.

What I will probably do going forward is create a reminder for myself via Backpack or iCal each month to A) check on the backups in Amazon S3, and B) clean out tmp/backup in the rails apps, especially ones where the backups push 1, 2, 3 gigs a pop.

I’d like to give a special thanks to those who commented on the release of backup_fu—especially for all of the positive feedback.

In this release, the following features were added:

- PostgreSQL dump support
- Ability to supply your own custom MySQL dump options (config key 'mysqldump_options')
- Backup_fu now checks your ENV variables for AMAZON_ACCESS_KEY_ID and AMAZON_SECRET_ACCESS_KEY;  if found, it uses these instead of requiring them in config/backup_fu.yml.

Since this is a minor release, there’s probably no need to update if you’ve already got v1.0 working like a charm.

For those who haven’t played with backup_fu, click here for the full scoop.

Thanks also to Toby Hede and Ralph Churchill for submitting patches.

Toby’s added the ability to support user-supplied mysqldump options; Ralph’s for PostgreSQL dump support.

Also hat tip to nick for suggesting the ENV variable support.

Redonkulously easy, that is.

Your rails app has been humming along nicely for several months, then suddenly it hits you, wait, are we backing up our database / uploaded files yet?

Well, of course you are. But for those who prefer to live on the edge, launching an app without a well-articulated backup strategy, I give you backup_fu.

It’s a rails plugin, invoked via rake tasks. After installing the plugin and modifying a few lines in a config file, you can have your application backing up both its database and static files to Amazon S3 within minutes.

Update: backup fu does now support PostgresSQL. See this post for the 411.

Installation

Grab the only dependency, the aws-s3 gem (if not installed already):

sudo gem install aws-s3

And the plugin:

ruby script/plugin install http://backup-fu.googlecode.com/svn/backup_fu/

Configuration

Generate the default config/backup_fu.yml file with:

rake backup_fu:setup

You’ll need to modify at least these four lines in config/backup_fu.yml:

# The app_name is used as the backup filename prefix
app_name: replace_me
# Note: please create this bucket (whatever yours may be) externally first:
s3_bucket: some-s3-bucket
aws_access_key_id: --replace me with your AWS access key id--
aws_secret_access_key: --replace me with your AWS secret access key--

If you’re on OS X, the excellent S3 Browser will have you creating S3 buckets, browsing them, and uploading/downloading to them within minutes.

Jets3t is a Java-based S3 browser and should do the trick on win32/etc systems.

Basic Usage

To dump your database (to RAILS_ROOT/tmp/backup):

rake backup_fu:dump

In production environments, of course, you may have to do something like:

RAILS_ENV=production rake backup_fu:dump

To dump your database, then send the tar/gzipped copy to Amazon S3, it’s as simple as:

rake backup

This will place a file named something like ‘foo_app_2008-02-07_db.tar.gz’ into your Amazon S3 bucket (as specified by s3_bucket from your config file).

Configuration Options

If you bump into any snafus, the first thing you should do is enable verbosity via the config file:

verbose: true

See vendor/plugins/backup_fu/config/backup_fu.yml.advanced_example for the list of advanced configuration options. ( view online )

The most common issue that one might run into is ‘mysqldump’ not being in the user path.

To solve this do a ‘locate mysqldump’ or otherwise find its absolute path, and specify it explicitly in config/backup_fu.yml:

mysqldump_path: /usr/local/mysql/bin/mysqldump

Also see the README for more on debugging snafus and advanced configuration options.

Static File Backups

Backing up your database is great and all, but what if users upload files too?

Let’s say you’ve got a directory RAILS_ROOT/public/static where all of these files reside.

In our fictional example, this directory is really a symlink to /apps/foo/static. So we specify this as the ‘static_paths’ key (again in config/backup_fu.yml):

static_paths: "/apps/foo/static" 

Let’s say we also wanted to backup ”/apps/foo/user_images”.

Multiple target static directories can be delimited via whitespace:

static_paths: "/apps/foo/static /apps/foo/user_images" 

First let’s try dumping these static directories (with full contents) into a tar/gzipped archive in RAILS_ROOT/tmp/backup:

rake backup_fu:static:dump

If that worked, let’s go for the full enchilada (dumping + uploading to S3) with:

rake backup_fu:static:backup

And for backing up both your database + static files (they will get uploaded as separate, distinct archives) in one rake command:

rake backup_fu:all

Phew.

Notes

While I had trouble with aws-s3 in its early days, I’ve since used aws-s3 (and backup_fu) to send a 4GB file to Amazon S3. VPSes or systems with much less memory might have issues backing up such huge files, though.

See the README for cronjob examples.

Pluginizing this code (which existed for a while in my apps, though not as a plugin) was inspired by Scott Patten who has written a similar kind of plugin.

Backup_fu does not erase old backup archives for you. This is left as an exercise for the reader. :) But seriously, it’s probably a good idea to check on your backups every month or so and do some pruning then, by hand if necessary.