search

garethgeorge / backrest

Backrest is a web UI and orchestrator for restic backup.
GNU General Public License v3.0
1.64k stars 43 forks source link

copy snapshots to other repos after creation #279

Open mariomare22 opened 6 months ago

mariomare22 commented 6 months ago

Note: if you have a question or want discussion please post in the discussions area.

Is your feature request related to a problem? Please describe. mostly overhead in backing up same plan to multiple repo

Describe the solution you'd like Possibility to copy a plan to multiple reps to decrease the overhead of backing up multiple times instead of having to create for each repo multiple plans. could be in selecting multiple repos in the following screenshot

image

Additional context feature provided and seen in autorestic Link to feature described.

garethgeorge commented 5 months ago

This is a pretty interesting idea. Dropping some ideas here re: implementation

I think this can be done as a CONDITION_SNAPSHOT_END hook that runs commands to copy snapshots to an additional configured repo in terms of the UX.

This is unfortunately complicated a bit by Backrest's data model as today it's assumed that the plan ID is singleton across all repos, but if you clone a snapshot that snapshot will be added to a different repo with the same tags and will be imported incorrectly. That's probably a bug that needs some thinking about.

Overall I'm definitely open to supporting this.

garethgeorge commented 5 months ago

Patched the issues related to backrest's data model in https://github.com/garethgeorge/backrest/commit/8a43f86f357b7471a22ba3e0cabed86f23d85af2 , all queries now respect both the repo and plan that an operation belongs to.

Janhouse commented 4 months ago

I think that having ability to somehow select multiple repos for the same plan is an important feature to have and it would be a better approach.

restic docs warn that copying snapshots has some drawbacks.

And being able to back up locally and remotely is quite important. Currently you have to duplicate the plans to achieve this.

Others suggest using rclone sync to duplicate repos, but if the main repo gets damaged, the clone will have the same issue, so at the end of the day it is better to run the backup twice, even if it is a bit slower.

garethgeorge commented 4 months ago

Spent a bit of time this afternoon investigating here -- the main challenge I see is that two repositories need to be specified which includes the need to be able to pass credentials. This doesn't look like it's very well supported.

e.g. the docs https://restic.readthedocs.io/en/latest/045_working_with_repos.html#copying-snapshots-between-repositories here talk about this a bit, but it's not obvious to me how one would go about providing two different sets of AWS access keys, for example.

Janhouse commented 4 months ago

But look at the notes in the restic docs, it is not a good idea to use the copy command.

In my opinion it would be better if we could reuse the same backup plan for multiple restic repos.

Currently it is tied to a plan one to one, I propose making it one (plan) to many (repos). It would run multiple times but would use less bandwidth and be faster (than restic copy), and probably be safer (than rsync/rclone).