search

offen / docker-volume-backup

Backup Docker volumes locally or to any S3, WebDAV, Azure Blob Storage, Dropbox or SSH compatible storage
https://offen.github.io/docker-volume-backup/
Mozilla Public License 2.0
1.77k stars 80 forks source link

v3 breaking changes #80

Open m90 opened 2 years ago

m90 commented 2 years ago

This issue exists to track possible breaking changes that would make sense in a v3 (so I don't forget about them). This does not mean a v3 is going to happen soon. It could even be it never happens.

Remove EMAIL_* config

As all notifications are now sent using shoutrrr, the EMAIL_* interface is obsolete and should be removed.

Remove BACKUP_FROM_SNAPSHOT

The functionality of BACKUP_FROM_SNAPSHOT can be achieved by using and exec-pre and exec-post command.

Default to expanding variables in BACKUP_FILENAME

The option to do this has been introduced later on and false was chosen as a default in order to not introduce a breaking change. true would be the better and more helpful default.

Also consider doing this for all other configuration values.

Rename BACKUP_SOURCES

BACKUP_SOURCES points to a single location. It should be called BACKUP_SOURCE (just like BACKUP_ARCHIVE).

Remove exec-[pre|post] labels

The labels have more granularity now.

Remove BACKUP_STOP_CONTAINER_LABEL

The BACKUP_STOP_CONTAINER_LABEL setting has been renamed: https://offen.github.io/docker-volume-backup/how-tos/replace-deprecated-backup-stop-container-label.html

rpatel3001 commented 2 years ago

consider rclone as a universal backend per #65? Ideally combinable with multiple sources and multiple schedules.

m90 commented 2 years ago

Ideally combinable with multiple sources and multiple schedules.

Could you elaborate on how this would be different from what is already possible?

rpatel3001 commented 2 years ago

I don't use the config file directory so I didn't know how it works but after looking at it I think it should combine just fine if every environment variable can be set individually for each run. The only difference would be another var with a list of rclone remote:folder targets to upload to. I'd also like a var that controls syncing all archives or just the symlinked latest one (I have a space constrained VPS that I sync just my latest to).

It would also be nice if the backup executable was able to be called multiple time in parallel to allow the same cron expression in multiple config files. Or maybe have backup configs leave out the cron expression and then have separate schedule configs, each with a cron expression and a list of backup configs to run sequentially. That might complicate setting up the container too much.

m90 commented 2 years ago

It would also be nice if the backup executable was able to be called multiple time in parallel

As the packaging format of this tool is a Docker image, I'm not entirely sure if getting rid of the exclusive lock (which probably is possible, albeit makes things more complicated) is worth it. If you really need to, you could still spawn multiple containers from that image which run in parallel just fine.

If the backup command were to be unboxed and distributed as a binary, other constraints would apply here, but up until then I feel it's ok to keep the lock and the simplicity it brings.

rpatel3001 commented 2 years ago

What is it that requires the lock? Or, instead of refactoring to not require a lock, could you check the lock in a loop until some timeout, to support identical cron expressions?

m90 commented 2 years ago

The lock is there for two reasons mostly:

I think your idea of waiting for the lock to become available instead of failing hard is a pretty good one though, so that should be a workable solution that allows reuse of cron schedules while still meeting above requirements.

m90 commented 2 years ago

I added support for identical cron expressions in https://github.com/offen/docker-volume-backup/pull/87 - I'll have another look at this myself tomorrow or so, but in case you have any feedback on that @rpatel3001 feel free to chime in.

rpatel3001 commented 2 years ago

looks good to me

m90 commented 2 years ago

Support for identical and overlapping cron schedules in now supported in v2.15.0