Shopping for a back-up program

In my latest blogs I have been describing how I have been exploring a way to back up my over 50,000 files to an off-site service so they will be secure in case my laptop and my local external backups are destroyed.  I explained why I chose Online Storage Solution (OLS-CS) to use for this off-site storage of my backed-up files.  I also explained why I needed a back-up program to handle the backups.  Copying files to the OLS-CS servers wasn't a satisfactory solution because later changes, additions, and deletions needed to be tracked and managed.  For that, you need a back-up program.

I had tried FoldersSynchronizer by softoBe and Personal Backup by Intego, but was having problems because when the connection was lost between my computer and the Network Drive on the OLS-CS servers, the backup just stopped, and I had to start the backup from the beginning over and over.

So I went out on the internet to see if there were any back-up programs that advertised their ability to maintain a connection or resume the backup after a connection is restored.

There are A LOT of backup programs out there.  Here is a list of a few that I looked at and a brief comment on each.

SmartBackup - $15 - Advertises itself as a fast and lightweight backup application for OSX, ready for Mountain Lion.  It appeared to be too simple for what I needed.  The program assumes it knows what you want when you are setting up the back-up parameters, leaving fewer things that you can change.

File Synchronization - starts at $15 - Appears designed to do synchronization only, not backups.

xTwin - $99.99 - Though expensive, this looks like a very good program.  It has gotten a couple really bad reviews, but those were for earlier versions.

Carbon Copy Cloner - $30 - Good reputation.

IBackup - Monthly fee based on amount of storage - This is a total on-line back-up service which includes storage and sharing of files. Not what I was looking for.

Mathusalem by softonic - No longer supported

SilverKeeper by LaCie - No longer supported

Of all the programs I looked at, the name I was most familiar with was Carbon Copy Cloner.  It has received glowing reviews for the past several years.  Being a contrarian, I had resisted getting it, but I decided that it was time to see if it lived up to the hype.  There is a 30-day free trial period.  After that it costs $39.95.



I thought I would start with backing up my Desktop folder.  There are only about 5500 files in that folder, and I thought it would be a good test of Carbon Copy Cloner (CCC).

This is a very comprehensive program with lots of options.  During the trial period, a window opens when you launch the program that reminds you of when the trial expires and gives you the option of continuing to use the trial or purchasing the program.

The main window allows you to select the source and the destination.  You can either choose to copy an entire disk to the destination or a particular folder.  The destination can either be a disk or a folder on a disk.  Since I was copying the Desktop folder, which resides inside the "bgmason" folder, which resides inside "Users" folder on my hard drive, I have the same folder structure created on the Network Drive.  I created a "Users" folder on the Network Drive, and then inside that I created a "bgmason" folder.  At this point, because of the previous work I had done making a backup of my files (see previous blogs), I had a "Desktop" folder and a "Documents" folder inside the "bgmason" folder on the Network Drive.

Once you make those selections, you have the ability to deselect files and folders that you don't want copied.  Even though they are small files, it doesn't make a lot of sense to copy all of the .DS_Store files, for example.  By deselecting them, there are that many fewer files to copy and it will somewhat reduce the time involved in making the backup.

On the destination side, you let CCC know what kind of media your backup will be made to.  You also are given options about whether or not to delete files on the destination that don't exist on the source or to save those files in an archive.

Finally, there is a button for further customizing the settings concerning data that already exists on the destination.  This includes a button called "Advanced settings..."  This is where I ran into trouble.  One of the selections is for removing files "on the destination that I have excluded from the backup".  It goes on to explain, "If an item exists on the destination (e.g. from a previous backup), and you have deselected that item from the list of items to be copied, that item will be removed from the destination."  Since I was deselecting some files, including the .DS_Store files, I thought I should select this.  That way, even though they currently existed on the destination because of my previous efforts to copy my files using FTP and the Finder, they would be removed.

All of these settings are remembered by the program so that the next time you select the same source and destination, you will not have to remember your previous settings.

So I started CCC off on its project to copy all the files and folders in my Desktop folder to the Network Drive.  This was taking some time, as I expected it would, and I proceeded to do other things on my laptop as the backup was proceeding.  But I glanced up at one point, and noticed that all of the files and folders in my Documents folder on the Network Drive were disappearing!  All that copying it had taken me days to accomplish using FTP and the Finder was being flushed down the drain!  I stopped CCC and assessed the damage.  I still had all the folders inside the Documents folder whose names started with A-C.  But the folders with names starting with D-Z were gone from the Network Drive.

In analyzing why this happened, I finally realized that when I said to remove files on the destination that I excluded from the backup, that this did not refer to the files and folders inside the "Desktop" folder, but rather the files and folders inside the "bgmason" folder.  Since I hadn't told CCC to copy the "Documents" folder, it commenced deleting it.

Needless to say, I was very discouraged at this point at decided to quit for the day. I would reconsider my approach, and then tackle the problem after I'd collected my thoughts.

Continuing saga of trying to back up files off-site

If you have read my previous blogs you know that I have over 50,000 files totaling over 30 GB that I want to back up to an off-site location. Because of the large number of files, I decided that most of the back-up services offered out there would not suit my situation. Those services are good for handling collaborative projects or for backing up particular "important" files and folders. But they are not set up to work with large numbers of files. I learned that what I needed was a "file storage" service.

After settling on Online Storage Solution (OLS-CS) to be my file storage service, my next task was to get my files copied to their servers.

I found that I could use the Finder and drag and drop files and folders that are located on my computer's hard drive to the Network Drive opened when connecting to OLS-CS. Or I could use an FTP client software to copy the files. This took a long time to move all the files in my Documents folder. Working on and off, it took about 5 days.

But once I had done that I realized that whenever I made a change to a file, or added or deleted a file, I would have to remember what I had done so I could copy those files to the OLS-CS servers. That would require a lot of effort and would be prone to errors. I needed a back-up program that would identify those incremental changes to my files and folders and copy the changed items to the Network Drive.

If you read my blog of July 23, you know that I had originally tried to use the back-up programs I own to get the files copied over, and that I ended up using my FTP software, Fetch, and the Finder instead. But I now knew that I had to be able to use a back-up program. So I tried once again.

The backup program that I have been using the longest is FoldersSynchronizer by softoBe. It has been around a very long time and is very easy to use. When I tried to use it, however, I found that the connection to the Network Drive gets dropped every once in awhile. It may appear to be connected when you look at the Network Drive in the Finder. The Finder wouldn't change appearances if the drop was only momentary. FoldersSynchronizer, however, stops the backup. It apparently has no way of resuming the upload once the connection is broken. In my research on the Internet, apparently the technology exists for resuming downloads, but it is much more difficult to resume an upload.

Another thing I began to realize as I started trying to use a back-up program is that in order for the program to do an incremental backup, it has to keep track of what existed before it begins doing the backup. So the total time it will take to do the incremental backup depends on how long it takes to scan the files to see which ones were changed, which ones are new, and which ones are no longer there.

When I saw that my old stand-by, FoldersSynchronizer, wasn't going to work, I decided to try a program I bought more recently, Intego's Personal Backup. I tried using it for several days. It seemed to make progress, but then it would quit. Each time it scanned my files to build a database of the files it would need to change. That process took three hours. Then it would start copying files to the Network Drive until the connection was broken, at which time it would simply stop copying. There is no error message. You just have to look at the information being provided and notice that nothing is changing. Each time you restart, there are fewer total files to copy because in the meantime the vast majority of them have not changed. Gradually you will get it down to the point where everything has been copied, and the only thing that will need to be updated are the files that you actually added, modified, or deleted during the period since the last backup.

Since I had already copied all my files to the Network Drive using FTP and the Finder, I couldn't figure out why FoldersSynchronizer and Personal Backup were finding that the files had changed and they had to copy the files all over again despite my previous effort. I eventually noticed that the creation, modification, and last modified dates were all being changed! When these programs copied the files to the Network Drive, the old dates were not being retained! Instead all the files were being given the date they were copied to the Network Drive.

I fired off a quick email to Support for FoldersSynchronizer, but never received a reply. My research on the internet tells me that this is a common problem and it is pretty much a mystery as to why this happens.

In my next blog I decide to go shopping for a backup program that won't fail when the connection to the Network Drive breaks. Stay tuned.

The latest update on my attempts to back up files off-site

If you read my last blog you know that I thought I had a solution for backing up my files off-site. But for the last couple weeks I have been trying to optimize the solution I found and get it to work the way I think it should. That is why I haven't posted anything since that last blog.

I want to recap to lay out where things stand as of now. (Sorry to repeat some stuff, but you may not want to go back and read my previous posts.)

How to move your back-up copies off-site

"Experts" agree, it is a good idea to have a backup of your files off-site so if your house or office is destroyed, you will still have a copy of your files. I suppose you could limit this off-site backup to just your "important" files, but what is "important" and what isn't? Making that decision could take more time than simply going ahead and making the copy.

There are several ways to create this off-site copy. If you are a business, it may be easier because you may own several locations, and you can just have copies on multiple hard drives in the various buildings you own. For a private individual, however, it isn't that easy.

If you are that private individual, you could make a copy of your computer's hard drive on an external hard drive and store that external drive somewhere like a relative's or a friend's home, or even put it in a safe deposit box. But then there is the problem of periodically retrieving the back-up hard drive from its off-site location so you can update it with the latest files.

I hadn't come up with a good, easy way to accomplish this goal of having an off-site backup. But then all this talk of the "Cloud" lead me to believe that I could back up my files to a server "out there".

As I said in my previous blog, for some time I have been using a service provided by Nomadesk. However, it was always a struggle using their service and I never was able to back up everything I wanted to. So I went looking for a better service.

What is being backed up

My situation is this. I have over 50,000 files on my MacBook Pro laptop totalling about 30 GB. My goal has been to copy all of that to an off-site server. I don't want a solution that provides automatic syncing of my files to the Cloud. Those types of solutions seem to be designed to sync those "important" or "essential" files. Apple's iCloud service, for example, is designed to sync your mail, music, photos, and iWork documents. It is not designed to sync documents you scan, or documents created in other applications like Microsoft Office or Photoshop, etc. Other services are designed for collaboration, so any files associated with a particular project would be backed up or synced to the service and then all of your collaborators on the project would have access to those documents.

I decided that the terminology for what I am trying to do is file storage. I want a service that will store copies of all my files on their servers and provide a way for me to recover those files if needed after some sort of disaster, or simply provide access to the files from my iDevices whenever I am away from my laptop.

As described in my previous blog, the service I found that I thought would provide what I was looking for is Online Storage Solution (OLS-CS). They provide unlimited storage for your files which you can access either through their web application which they call "File Manager", or by connecting to their server as a Network Drive which appears in the Finder on your Desktop, or on your iDevices through an App called "AjaXplorer". What I have been struggling with over the past couple weeks is how to copy the files from my laptop to the OLS-CS servers.

When I posted my last blog on July 23, I said I had been able to get all the files in my Documents folder copied. I accomplished this by using Fetch, my FTP client, and the Finder, simply dragging and dropping files from the list of files in the Documents folder on my hard drive to a Documents folder I created on the Network Drive.

Simply copying files is not the solution

This might seem to have accomplished my goal. I now had an off-site backup of my files.

However, there is one problem with this way of backing up your files. The minute you change, add, or delete a file, you need to go to your FTP software or the Finder and copy the files that were changed to the folder on the off-site server. 

You definitely don't want to do this manually. It would be just to difficult to keep track of what you worked on, created, changed, and deleted during the period since the last time you copied your files to the backup.  So you need a back-up program that is going to do this for you, a program that will do incremental backups based on what has changed in the source folder. So this is what I've been studying for the past week. What software is out there that will be able to do this for me given the large number of files involved and the fact that I'm trying to do my backup to a Network Drive.

In my next blog I will try to explain what I have learned.