[Feature Request] Differential Backup

[jochen-01]

Is your feature request related to a problem? Please describe it. I like to follow a backup strategy used widely in data management since decades: Differential backups of programs and data. Full backups requires much time and much storage.

System Information (please complete the following information):

• Device: Fairphone 4 (but does not matter)
• Android Version: 11
• ROM: (AOSP, MIUI, OOS...) stock
• App Version: 8.2.4

Additional Notes

I think it's clear what I want to do: Full backups let's say, every month. Between these full backups differential backups (perhaps) every day. From a programming point of view differential backups are much easier to implement than incremental backups, therefore I would suggest to realize just differential backups.

[hg42]

please answer these simple questions first:

• how many backup generations do you use currently?
• which additional percentage of storage would you spend for these backups?

[jochen-01]

Currently I'd like to use 2 generations, since I want to use 4 differential backups. One monthly full backup and 3 weekly differential backups. Perhaps I have to explain my whole backup strategy: I make make local backups on phones' MicroSD. Monthly I backup that MicroSD to a server using rsnapshot.

Additional storage (I think you mean "disk" space on backup target?). Making backups like that I think I can reduce the backups I have to keep. Thus the differential backup should not use much additional space.

[hg42]

Yes, with storage I meant the target storage system for the backup.

Is your main concern about backup size or about backup creation time (on the phone) or about transfer time to the server?

My questions were about getting expectations and reality in sync.

Your original "1 monthly full + 30 daily differential" strategy with for example an average of 10% for each incremental backup would take up 4 times of the space. The time is also questionable, because comparing the old with the new backup also takes time and involves reading all the data.

With 1 monthly and 3 weekly, we are more on track. Do you backup most of the packages or only a few? Is the server on your local network (NAS, home server, Workstation...) or Internet?

The usual strategies to overcome the compare problem use file time, checksums, hard links or file systems with snapshot capabilities. File time is kind of unreliable especially on the target storage, hard links are not available on most backup storages, snapshot file systems (like btrfs) are even more unlikely. All methods are file based, so you basically have a copy of the source file tree on your backup storage.

[jochen-01]

Transfer time to server is not a issue. I do that one a week by moving the card from the phone to a reader on the h I make backups of all packages.

I tried to sync my wishes with the capabilities of Neo Backup. ;-) But without differential backups it's not easy. In an ideal world I would make

• full backups weekly, keeping 2 of them on the phone. These are backed up to the host once a week. There I have hard links, reliable file modification time stamps and snapshot capabilities (by software),
• differential backups daily keeping 8 of them.

Notice that they are overlapping a bit -- just in case of delays in doing host backups.

For quick restore tasks I have the recent backups on the phone. For longer steps into the past I have snapshot backups for a whole year on the host.

[powerman]

@hg42 Any plans/progress on this?

My use case is similar: with Titanium Backup I get used to daily backups uploaded to Dropbox. It was very convenient because backup was done only for changed apps, and as most apps won't used/updated (and thus change) every day daily addition to backup was small enough to make it convenient to upload it to Dropbox every day. Also it was very convenient to have 3 last apk versions in backup for unused apps (because unused apps "change" only when they updated) - because sometimes updates result in a broken app, and for apps which are not used every day it's hard to notice this fact, so ability to restore 2 previous apk versions from backup (when you occasionally notice broken app) was really invaluable!

After switch to NeoBackup user experience has changed:

• Policy "last 3 backups" for apps which are not used every day (i.e. for most of apps!) effectively became "last 1 backup" because all 3 backups contains the same data.
• Daily upload to Dropbox became an issue: uploading several GB (because now daily backup include all apps, even if they didn't change) every day from phone isn't fast and convenient.
• I had to install 2 NeoBackup versions (prod and debug) to be able to keep both last 3 daily backups of app's data and last 3 apk versions (because new apps are installed or updated not every day and thus apk backups gets deleted in favor of last 3 data backups).

If NeoBackup will skip creating new backup in case there is no changes compared to previous backup this will solve all these issues!