Deduplicated backups (e.g. Borg or restic)

[bittner]

IIUC, the current feature set requires you to handle rotating the backups yourself.

• Whether you compress, encrypt and store to whatever backend, there will always be something like a tarball for every new execution of a backup process. Hence, you have to deal yourself with deleting old copies (and keeping selected ones, e.g. one every week for the last 3 months) to make sure you don't run out of disk space.
• Also, if there are no changes (be it in the database or the media files) there will still be a backup, which will unnecessarily occupy disk space and/or incur additional cost on a storage service.

These problems are solved by backup applications that do deduplicated backups, such as Borg or Restic. The latter is a Go implementation, the former a Python tool.

Would it be realistic to try and integrate those tools in django-dbbackup? Calling them as external binaries would likely be sufficient.

[bittner]

runrestic is one of the more promising attempts to provide restic for Python.

[jonathan-s]

@bittner What could this package do better than runrestic does?

[bittner]

Integration with Django for both the database data and the static files?

Really, all the restic wrappers are just there because restic itself doesn't provide a way to maintain a configuration. The "promising" from above relates to merely the rather poor state of the other restic-related packages. :pensive: At least, for what I can see.

[jonathan-s]

If you'd like to provide a patch for this it would be welcome. If you take this on, please do describe the approach you'd like to take so that we can agree that it sounds sensible for both of us :).

[ZuluPro]

Hi @bittner dbbackup has a simple retention option: DBBACKUP_CLEANUP_KEEP and DBBACKUP_CLEANUP_KEEP_MEDIA In number of backups

If you add the --clean option to the commands, it will apply this retention by guessing the date from fliename.

https://django-dbbackup.readthedocs.io/en/master/configuration.html#dbbackup-tmp-file-max-size