incremental backups?

[r2evans]

Do you have any interest in enabling incremental backups to the backup methodology? Especially with nextcloud, backups get hard and expensive, and incremental backups are widely seen as a preferred option.

Since the current method uses tar, the solution can either:

1. handle things manually with --listed-incremental=<some_state_file>, then using --level=0 for full backups and --level=1 for incremental backups (relative to the most recent level 0 backup); or
2. include and use the tar-provided backup and restore scripts (perhaps overkill for this simple configuration).

I'm proposing simple logic that would do something like:

• define a "period" to force full backups; any backups performed inside of this period would be level-1 incrementals based on the most recent full, meaning that only the full and most-recent incremental would be required for restoration; I suggest a default of BACKUP_FULL=604800 (seconds in a week)
• if there has not been a full backup within the last period, perform a level-0 (full) backup
• if there has been a full backup within the last period, perform a level-1 (incremental) backup

Since the backup file you output contains several tar files, this logic could be applied to the ocdata.tar file itself, and optionally to config.tar.gz (though this has very little benefit due to its small file size).

A future extension to this would be to save the diff for database.sql(.gz) in the incrementals. (Not suggested at this time.)

References:

• Incremental backups with tar, https://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html
• Levels of backups, https://www.gnu.org/software/tar/manual/html_node/Backup-Levels.html
• backup and restore script configuration, https://www.gnu.org/software/tar/manual/html_node/Backup-Parameters.html

[r2evans]

This might apply to docker-gitlab as well, if you were to take over the backup of repositories. Again, small steps first, but the space-savings will be significant for anyone (who does backups) but the smallest of users.

[r2evans]

I believe a more formal backup setup probably belongs outside of the docker container. I've combined all of my services (incl. gitlab, nextcloud) into a single incremental backup using duplicity. Without significantly more configuration, I get the more-efficient incremental backups I wanted plus some other options (e.g., encryption, remote backup host, etc).