Switching from SpiderOak to IDrive

I’ve been using SpiderOak ONE for my online cloud backups since 2012 … almost 12 years. I was grandfathered into an unlimited plan for $125/yr and had just under 5TB backed up to the service. On April 24, the service went down for “scheduled maintenance” to move to a new data center … and was down for two weeks! Worse, there was near-zero communication about the outage, no response to support tickets, affected a lot of people (see here and here for examples), and in fact is still causing problems for customers today (over a month later)!

My annual renewal was coming up at the end of May and this was now the second big service outage so far this year (a total of 20 days of downtime!). It became clear to me that SpiderOak no longer considers their ONE backup service important (in fact, the conspiracy theorist in me thinks they did this on purpose to get rid of ONE customers), so I started looking at alternatives.

I had some basic requirements based on my SpiderOak usage:

  • encryption (zero-knowledge preferred)
  • backup file versioning
  • runs on Linux (Docker preferred, or alternatively a native Synology NAS app)
  • reasonable cost (and low ingress/egress costs if applicable – I noticed some services charge for that on top of the storage itself!)

Nice to haves would include:

  • de-duplication of backed up files (to save space)
  • multi-platform (Windows, Linux)
  • mobile app

After researching services like Crashplan, Backblaze B2, IDrive, pCloud, and more complex roll-your-own solutions like Borg, RClone, and Kopia, I selected IDrive.

PRICING

Using a promo code, I signed up for the 10TB plan for $5 for the first year. After the first 12 months that 10TB will cost me $150/yr, slightly more expensive than what I was paying for an unlimited SpiderOak account, but since I’ll only be using about 50% of my IDrive quota initially, the “unlimited” part didn’t really matter for the sake of comparison. Like SpiderOak, you can backup any number of devices to your account as long as you stay within your quota.

ENCRYPTION AND SECURITY

IDrive uses 256-bit AES data encryption before transmitting files from your computer to your cloud storage. When setting up your account, you have the choice to use a default encryption key or a private encryption key. If you want to be able to share files in your IDrive account you must use the default key. I guess I can understand why some people would want that but I used my own private key. The risk here, of course, is that if you forget your private key, IDrive can’t help you get access to your data so you’d have to back up everything again. You also cannot change your private key without re-uploading (everything).

It’s not clear to me exactly how much zero-knowledge is involved here. According to the IDrive FAQ, the “encryption process encrypts the data contained within the file and not the file-name itself. Certain metadata related to files including file thumb derivatives may not be encrypted. These metadata and derivatives are necessary for faster display and search.” To be honest, though, I really have no idea if the same thing applied to my SpiderOak backups.

Decryption occurs locally on the client using the IDrive desktop or mobile apps. But if you want to use the web site to access your files, you have to enter your private key which means that key is “temporarily” stored in the intermediate server that is handling the decryption in order to let you see your files through the browser. So technically at this point IDrive has access to that key, just something to be aware of (SpiderOak had the same issue when using the browser interface).

I also like that IDrive supports 2-factor authentication when logging in (even in the desktop apps), something SpiderOak never seemed to implement.

INITIAL LOAD

The hard part about switching backup providers is the initial upload to transfer all of your data to the new service. When I switched from JungleDisk to SpiderOak back in 2012, I had 20mbps upstream and it took just over 5 days to upload ~75gb (aww, how cute)! Then it took me a month in 2014 to upload 1TB. Now I have 500mbps upstream, but 5TB to upload so I was looking at probably more than a month (and there are reports that IDrive’s upload speeds are not great). Luckily, IDrive offers the IDrive Express service: they send you an external hard drive, you use the IDrive app to backup everything you want locally on the drive, then mail it back to them and they load it into your account directly. This should presumably should take a lot less time than uploading everything over the internet.

Once I received it in the mail, I connected the Express drive directly to my Synology NAS and then used the Express Backup option in the app to back up all the folders I wanted. Even with the direct USB 3.0 connection, it took 11 days to copy 4.48TB of data to the external drive! In comparison, when I did a full NAS backup before migrating from my older Synology DS214 to the 720+, the 3.2TB copy to an external drive took only 2 days. I can only speculate that the added encryption is what slowed down the copy. Since I’m using a private encryption key, all of the files I copied to that drive were encrypted: I checked the folder structure and confirmed everything had random characters for names. This seemed to conflict with what I saw in the FAQ (see the Encryption section above), but I was glad to see it, especially since I was a little wary of sending a hard drive containing all of my data in the mail.

Once the backup was done, I mailed the drive back to IDrive and received a notification that my data was being copied into my cloud storage account. It was another week before that copy was done so start-to-finish it took just under 3 weeks to get my initial backup loaded. Definitely easier and faster than doing the intial load over my internet connection, but I was hoping for it to be slightly faster.

You can also use the Express service to restore your data locally, basically the reverse procedure: IDrive will copy everything in your account to an external drive and then send it to you so you can restore locally instead of downloading over the internet. The service is free once a year for personal customers.

BACKUP BASICS

While you can use IDrive as a synchronized “cloud drive” like OneDrive or Box, its primary use (for me, anyway) is backups. IDrive will do a full backup of a file that doesn’t exist in your cloud account yet, and an incremental backup for a modified file (changes only). A backed up file will retain up to 30 previous versions of the file and those versions don’t count against your quota. A “snapshot” feature in the apps uses that versioning behind the scenes to help you quickly find all files from a specific date (for example, to restore everything from before a ransomware attack). SpiderOak didn’t have a limit on file versions, but you did have to occasionally run a manual “purge” process to delete old versions specifying granular options like “keep all versions from the past 7 days, then one version each from the past four weeks, and then one version from the past 6 months beyond that.” I didn’t really use that level of versioning, though.

Another one of IDrive’s advertised features is true archiving: “With true archiving, IDrive keeps all of your backed up files, including those which are no longer present on your computer, available in your IDrive account. That means that unless you have deleted a file from your account manually or through the automated Archive Cleanup process, you’ll always have access to it. In other words, IDrive truly archives the files you backup.” This is a big difference from SpiderOak: if a file was deleted locally, it was moved into the trash in my SpiderOak account and then (eventually) purged. Of course, there’s also a downside to true archiving: since nothing is deleted, it’s up to you to do occasional archive cleanups (or scheduled periodic cleanups) to make sure you don’t go over your quota. And those cleanup processes don’t send files to the trash, they delete/remove the files from your account, so be careful!

Items in the trash will remain there for 30 days before being removed completely from your account. And again, there’s an important distinction here in that items are only moved to the trash when you delete them manually (through one of the IDrive apps) from your cloud account. If you delete a file locally on your PC, that file stays where it is in your cloud backup due to the true archiving feature. If you run an archive cleanup, those files are removed from your account, not moved to the trash (I sent in an enhancement request about this) and it’s an all-or-nothing process. If you only want to clear out specific files, you have to do it manually. Finally, full machine backups aren’t covered by the archive cleanup process. With 5TB free still I’m in no danger of needing to worry about this quite yet, but it’s just another thing to be aware of and I still need to play around more with all of this to really understand how it works and how it impacts my backup strategy.

A note about backing up large files (and this was the same with SpiderOak): the IDrive client first makes a copy of the file so that it can be encrypted before uploading. As a result, you cannot back up a file larger than the available free disk space you have on the machine running the client (although this doesn’t seem to apply to a disk clone/system image operation, which is a block-level backup).

CLIENTS AND BACKUPS

Like SpiderOak, IDrive has apps for Windows, Mac, and Linux machines but unlike SpiderOak, IDrive also has mobile apps  for iOS and Android devices. While waiting for my Express backup to move the majority of my data into my new IDrive account, I started playing around with the apps and learning my way around the new system.

Synology NAS

IDrive has apps for QNAP, Netgear, and Asustor NAS devices, as well as a Linux package for servers. The Syno app was the first IDrive app I encountered since I used it for my Express backup (see above). Also, it’s the most important app for me since this is where my primary backups are run. The native Synology app from IDrive actually uses a browser interface so it required the installation of the Web Station and PHP 8.0 packages. When you click on the IDrive icon in the Synology UI, it actually just opens a new browser window connected to the Web Station web server that hosts the pages for the app. You first log in using your NAS credentials, then your IDrive credentials (including 2-factor authentication, if enabled) and finally the private encyption key (note: although the Synology NAS app uses this browser interface I confirmed with IDrive support that the IDrive app running on the NAS is the client handling the decryption, so there’s no “leakage” of your private key in this situation compared to logging into the IDrive web site directly).

In the app you can create new and edit existing (up to five) backup sets to back up files to your IDrive or a USB drive locally attached to your NAS (like how the Express Backup works). You can also (obviously) restore files from your IDrive cloud account back onto the NAS. I like that you can actually see the IDrive backups from all of your devices in the app, so if I wanted to restore a photo from my wife’s iPhone through the Synology app to my local PC, I can do that. The scheduler tab allows you to control when backups are executed and also send e-mail alerts (always, or only on failure). You can view all of your backups logs through the app (or download them) and I have to say the logs are much nicer (with one exception) and more user-readable compared to SpiderOak.

Under Preferences you can set up file and folder exclusions with either full path names or partial names. Just like SpiderOak, these exclusions are global so you can’t exclude .tmp files from just a specific folder at this level, it’s all or nothing.

I sent in a few feature requests specific to the Synology app (and they may apply to the other NAS/Linux apps as well):

  • rename scheduled backup sets (without needing to delete entire backup set and re-creating it)
  • temporarily disable scheduled backup sets (the only way to do this is to keep editing the schedule and pushing it into the future)
  • delete files from cloud backup (unlike the other apps, there’s no “delete” icon on Restore tab)
Windows PC

I only ran the SpiderOak client (in a Docker container) on my NAS, not my PCs, but I installed the IDrive Windows app just to see how it would work. Similar to the Synology app, you can create, edit, and schedule your backup sets but unlike the NAS you’re locked into a fixed set of available backup sets (which cannot be renamed). You might not use/schedule all of these, but they’re your only option:

  • Default BackupSet_<user>: the main backup set for the current user that you set up on the Backup tab under “Backup files to my IDrive account”
  • LocalBackupSet_<user>: created on the Backup tab under “Backup files to Local, Wi-Fi or Express device”
  • MappedDriveBackupSet_Online_<user>: used if you enable “Mapped Drive Setup” under Settings to back up mapped drives
  • IDMachineBkset[Local]: used by “Entire Machine Backup” for backing up to a local drive (i.e., a mapped drive on a NAS)
  • IDMachineBkset: used by “Entire Machine Backup” for backing up to your IDrive account
  • IDSystemImageBkset: used by “Disk Clone” for a complete system image

Confused? Yeah, it seems more complicated than it needs to be. I’m not sure why you can’t just create and name backup sets like you can in the Synology app.

Most of the features are similar to the NAS app: you can see the backup sets of your other devices, set exclusions for folders/files, etc. Other differences from the NAS app are the Cloud Drive and Continuous Data Protection (CDP) features (neither of which I’ll be using). Cloud Drive is like the SpiderOak Hive, OneDrive, or Box like I mentioned in the Backup Basics section above. You can use this to keep files in sync across devices or share files with others (but not if you’re using a private encryption key). CDP will continually monitor the files in your backup sets and then back up a new version immediately after a change. We’re an Office 365 family so we all run OneDrive on our machines and that’s handling the versioning for us so I don’t have plans to use the IDrive Windows app that much.

A note here about disk cloning: a full machine backup set will fail if you have BitLocker (or any whole-drive encryption) enabled on your machine (“Sytem Backup failure:: OS drive is bit locked, Skipping the Backup.Reason:Shadow Copy cannot be created for disk”). A system image however, will work. The downside of a system image is that the size of the image is the same as the size of the disk being imaged so the image of a 1TB drive will take up 1TB of your IDrive quota even if only 300gb of that 1TB drive is in use. A full machine backup of that same drive would only take up 300gb of your IDrive quota, but with BitLocker enabled, that’s not an option.

The feature requests I sent in for the Windows app:

  • BitLocker support for Full Machine Backups
  • more granular/flexible scheduling options: there’s no monthly option (like the Synology app), no option for every x hours (instead of just hourly) or every x hours on weekdays only, etc.
iPhone/iPad

SpiderOak used to have an iPhone app, but it was so slow it was basically unusable for browsing files in my backups so I never really used it. The IDrive iOS app isn’t fancy, but it is pretty snappy and it has some basic backup features to boot. Like Microsoft’s OneDrive mobile app you can turn on auto-upload for your photos and videos, but the app will also back up your contacts and calendar. Most people with iPhones probably use iCloud for that but a second option doesn’t hurt. It’s a manual process though: the backups won’t run in the background, you have to launch the app and tap Backup. You can also use the mobile app to browse the backup sets from all of your other IDrive devices, view supported files (like Office docs and PDFs), and download photos and videos (even across mobile platforms).

One problem I ran into: if you use the “Report Problem” link (under Settings), it assumes your default mail app is Apple Mail. If you use a different app (like GMail, which is my primary app), the process won’t work properly. I found that out while I was trying to work with support on a bug I had found: if you have multiple iOS devices backing up to your IDrive account, they will all be named “iPhone” or “iPad” instead of the name of the device like the documentation implies. I ended up sending support all sorts of screenshots and doing a chat session and supposedly development is going to look into the issue. Update 10/06/2024: this has been fixed in v5.3.7 – you can now swipe on a device name and select “Nickname” and change the default so that your iOS devices all don’t have the same name.

Dashboard

Despite my concerns about IDrive’s access to my private encryption key when using a browser, there are actually some cool features you can access via the IDrive web site. Just like the client apps, you can browse the cloud backups from all of your devices and download (or delete) files. If you’re using the Cloud Drive feature you can see those synced files as well. Beware of emptying your trash from the web interface though, there’s no confirmation dialog like there is on the desktop apps!

The cool feature is the dashboard where you can see the other PCs (but not the NAS or mobile devices) connected to your account and manage the IDrive client remotely. In addition to pushing app updates, you can pretty much do anything to the IDrive app that you could do if you were sitting physically in front of the machine (manage backup sets, change schedules, update settings, etc.). Pretty handy if you’re dealing with remote machines (like my daughter’s laptop at college).

I sent in a few more features requests which, while not specific to the web UI, are more general so I’m listing them here:

  • suppress (or make it an option to supress)  the “excluded files” from backup logs; my list of excluded files is pretty extensive (.tmp, .itc2, etc.) so the log files are large and 90% of them are the same list of excluded files each time which makes it difficult to just quickly scan the log file
  • change the archive and period cleanup processes to send deleted files to the trash instead of just deleting them from the cloud backup (that would allow for 30 days to recover the file from the trash before it’s permanently removed)
  • change the e-mail subject line to include the machine name and job name (or make it customizable) instead of “Backup Status Report [<account>] [Backed up file(s): 628 of 1957642] [Successful Backup]” which isn’t helpful when scanning subject lines

SUMMARY

Although I’m just over a month into my IDrive experience, it’s only been a week since my Express Backup completed and I was back to “normal” with regards to my backups so I’m sure I’ll still be discovering things about the service going forward. I’ve sent in a lot of tickets and feature requests to support and they seem pretty responsive (much better than SpiderOak, even before the latest fiasco). I will say that I’ve noticed the upload speeds do seem slow compared to SpiderOak (and I’d read similar complaints when doing my research). This doesn’t really matter to me since typically my scheduled backups run overnight (last night, for example, my primary backup job (on the Synology NAS), ran for 2.5 hours and backed up 1.8gb, 628 files).

So far, IDrive has been worth the $5 I paid for my first year of 10TB. 😉 Let’s see how it holds up …

4 Comments

    • IDrive Support’s response (not much to action on here):

      The speed of the restoration completely depends on the following factors:

      1. Network Traffic at the time of restoration.
      2. Bandwidth provided by your Internet Service Provider.
      3. The speed of your Internet connection.
      4. The total size of the restore set.

      IDrive is designed to use your upload or download speed provided by your ISP. IDrive does not impose any bandwidth restriction on upload or download speed. We don’t impose any bandwidth restriction for user accounts, this is the same across all data plans. So we will not be able to increase or decrease the backup speed from our end.​

      If you still feel it is doing a slow restore, send us the logs by following the below steps:

      To send error reports to IDrive support,
      Login to the Synology Backup App.
      Click the username on the top-right corner, and click ‘Send Error Report’.
      Fill in the required fields and click ‘Send’.

      You may click on the following link, select the server location as Oregon and provide us a snapshot of the upload/download speed result.
      http://www.speedtest.net

      Thanks,
      Your IDrive Support Team

Leave a Reply

Your email address will not be published. Required fields are marked *