Backing up Windows Server 2008 files nightly to offsite Debian Linux box using cwrsync (rsync)


We’ve had endless problems trying to get offsite backups of our office Windows Server 2008 data. The batch script we designed would run fine in testing, but then fail when run as a scheduled task.

This post is designed in the hope anyone stuck with a similar problem might find our solution useful (it seems like we finally got it working!).

Again, you will need to be quite techy already for this post to make sense.

First, a bit of background:

We do nightly full backups to a USB hard drive that sits in the office data room

So we just want to back up data on our shared drive and our Users drive (we have these on a partitioned D drive – you might have yours on your C drive)

We have an offsite server that runs Debian Linux (so if you are backing up to another Windows box this post won’t help – but I would guess any Linux box would be fine).

Some of the things we did:

  • make sure you do everything as an administrator on the Windows box
  • install cwRsync (we did it into in C:\Program Files (x86)\cwRsync)
  • get rsync running properly on the remote backup server
  • set iptables on the remote server to allow ssh connections from our office
  • set up a public-private key, installed in C:\.ssh
  • get ssh to work
  • get rsync to work when it is run manually (as administrator)
  • set up a Windows scheduled task with ‘highest privileges’
  • make sure that the administrator that will be running the Windows scheduled task has “Full control” Windows security permissions over the files you want to back up (this will usually be fine – if the user is a member of the Administrators group)
  • make sure if you’ve got a shared network drive (we call ours the p drive) that your user’s virus checkers aren’t scanning files on this drive
  • make sure if you’re running a full backup, that you do the rsync-ing after this has finished

We then created a batch script – here it is:

SETLOCAL

SET CWRSYNCHOME=C:\Program Files (x86)\cwRsync
SET CWOLDPATH=%PATH%
SET CYGWIN=nontsec
SET HOME=%HOMEDRIVE%%HOMEPATH%
SET PATH=%CWRSYNCHOME%\bin;%PATH%
SET DESTINATION_USER=[user name on your backup server]
SET DESTINATION_HOST=[IP address of your backup server]
SET DESTINATION_ROOT_FOLDER=/home/%DESTINATION_USER%/BACKUPS/Windows_Server
SET DESTINATION_FULL=%DESTINATION_USER%@%DESTINATION_HOST%:%DESTINATION_ROOT_FOLDER%
SET EXCLUDE_COMMANDS=--exclude="**/My Documents/$RECYCLE.BIN/**" --exclude="**/Downloads/**" --exclude="**/$RECYCLE.BIN/**"

CALL :s_rsync pdrive
CALL :s_rsync Finance
CALL :s_rsync Users

GOTO s_end

:s_rsync

echo %DATE% > d:\%1\.sync
SET LOGFILE="%CWRSYNCHOME%/logs/%1.txt"
echo Starting backup of %1 > %LOGFILE%
echo %DATE% >> %LOGFILE%
echo %TIME% >> %LOGFILE%
echo Logs >> %LOGFILE%
rsync -a -v -v -v --delete --timeout=120 %EXCLUDE_COMMANDS% --chmod u+rwx -e "ssh -i c:\.ssh\id_dsa" "/cygdrive/d/%1" %DESTINATION_FULL% >> %LOGFILE%
GOTO :eof

:s_end
ENDLOCAL
ECHO "Done"

You will definitely need to tweak this script to your own requirements. It backs up D:\pdrive, D:\Users, D:\Finance – you will of course want to do something different.

The key bits in it that we’ve found useful are:

The SETLOCAL and ENDLOCAL commands in DOS mean that the variables we create only persist while we’re running the batch file

EXCLUDE_COMMANDS is a bit of a pain but we couldn’t get cwRsync to use an external file list of excludes. You will probably want to tweak this, but the list here is quite a useful start

We put a file .sync into the folder we’re backing up so that we can run a cron job on the backup box to check the time stamp of this file and make sure the backups have run.

The -v -v -v bit of rsync makes it create a full debug output – great for seeing what’s gone wrong (which it may well do!)

The –delete bit will delete anything on the backup server, if it’s been deleted on the Windows box

The –timeout=120 is a bit of an experiment, to try and stop the Windows scheduled task hanging if for some reason ssh doesn’t work

The –chmod u+rwx sets the correct permissions on the files on the backup server

The -e “ssh -i c:\.ssh\id_dsa” is important – it makes sure that when you are running as a Windows scheduled task, that the ssh bit of cwRsync can find the key

We dump all the output of rsync to a log file (we created a subfolder of the cwRsync directory called logs, but you could keep this anywhere).

GOOD LUCK!

Advertisements

About saasmd

I am an experienced software-as-a-service entrepreneur, based in London, UK. I love building interesting software businesses. My current venture is StorIQ, a platform to help bricks-and-mortar retailers manage their operations more effectively. This blog is a space to share low-level techie stuff that I think other people will find useful.
This entry was posted in Techy Stuff. Bookmark the permalink.

One Response to Backing up Windows Server 2008 files nightly to offsite Debian Linux box using cwrsync (rsync)

  1. Tom Lovie says:

    Thanks – this was helpful. Been struggling all day

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s