We’ve had endless problems trying to get offsite backups of our office Windows Server 2008 data. The batch script we designed would run fine in testing, but then fail when run as a scheduled task.
This post is designed in the hope anyone stuck with a similar problem might find our solution useful (it seems like we finally got it working!).
Again, you will need to be quite techy already for this post to make sense.
First, a bit of background:
We do nightly full backups to a USB hard drive that sits in the office data room
So we just want to back up data on our shared drive and our Users drive (we have these on a partitioned D drive – you might have yours on your C drive)
We have an offsite server that runs Debian Linux (so if you are backing up to another Windows box this post won’t help – but I would guess any Linux box would be fine).
Some of the things we did:
- make sure you do everything as an administrator on the Windows box
- install cwRsync (we did it into in C:\Program Files (x86)\cwRsync)
- get rsync running properly on the remote backup server
- set iptables on the remote server to allow ssh connections from our office
- set up a public-private key, installed in C:\.ssh
- get ssh to work
- get rsync to work when it is run manually (as administrator)
- set up a Windows scheduled task with ‘highest privileges’
- make sure that the administrator that will be running the Windows scheduled task has “Full control” Windows security permissions over the files you want to back up (this will usually be fine – if the user is a member of the Administrators group)
- make sure if you’ve got a shared network drive (we call ours the p drive) that your user’s virus checkers aren’t scanning files on this drive
- make sure if you’re running a full backup, that you do the rsync-ing after this has finished
We then created a batch script – here it is:
SETLOCAL SET CWRSYNCHOME=C:\Program Files (x86)\cwRsync SET CWOLDPATH=%PATH% SET CYGWIN=nontsec SET HOME=%HOMEDRIVE%%HOMEPATH% SET PATH=%CWRSYNCHOME%\bin;%PATH% SET DESTINATION_USER=[user name on your backup server] SET DESTINATION_HOST=[IP address of your backup server] SET DESTINATION_ROOT_FOLDER=/home/%DESTINATION_USER%/BACKUPS/Windows_Server SET DESTINATION_FULL=%DESTINATION_USER%@%DESTINATION_HOST%:%DESTINATION_ROOT_FOLDER% SET EXCLUDE_COMMANDS=--exclude="**/My Documents/$RECYCLE.BIN/**" --exclude="**/Downloads/**" --exclude="**/$RECYCLE.BIN/**" CALL :s_rsync pdrive CALL :s_rsync Finance CALL :s_rsync Users GOTO s_end :s_rsync echo %DATE% > d:\%1\.sync SET LOGFILE="%CWRSYNCHOME%/logs/%1.txt" echo Starting backup of %1 > %LOGFILE% echo %DATE% >> %LOGFILE% echo %TIME% >> %LOGFILE% echo Logs >> %LOGFILE% rsync -a -v -v -v --delete --timeout=120 %EXCLUDE_COMMANDS% --chmod u+rwx -e "ssh -i c:\.ssh\id_dsa" "/cygdrive/d/%1" %DESTINATION_FULL% >> %LOGFILE% GOTO :eof :s_end ENDLOCAL ECHO "Done"
You will definitely need to tweak this script to your own requirements. It backs up D:\pdrive, D:\Users, D:\Finance – you will of course want to do something different.
The key bits in it that we’ve found useful are:
The SETLOCAL and ENDLOCAL commands in DOS mean that the variables we create only persist while we’re running the batch file
EXCLUDE_COMMANDS is a bit of a pain but we couldn’t get cwRsync to use an external file list of excludes. You will probably want to tweak this, but the list here is quite a useful start
We put a file .sync into the folder we’re backing up so that we can run a cron job on the backup box to check the time stamp of this file and make sure the backups have run.
The -v -v -v bit of rsync makes it create a full debug output – great for seeing what’s gone wrong (which it may well do!)
The –delete bit will delete anything on the backup server, if it’s been deleted on the Windows box
The –timeout=120 is a bit of an experiment, to try and stop the Windows scheduled task hanging if for some reason ssh doesn’t work
The –chmod u+rwx sets the correct permissions on the files on the backup server
The -e “ssh -i c:\.ssh\id_dsa” is important – it makes sure that when you are running as a Windows scheduled task, that the ssh bit of cwRsync can find the key
We dump all the output of rsync to a log file (we created a subfolder of the cwRsync directory called logs, but you could keep this anywhere).