garethgeorge / backrest

Backrest is a web UI and orchestrator for restic backup.
GNU General Public License v3.0
1.72k stars 50 forks source link

Add Support for --skip-if-unchanged #417

Closed fortysix2ahead closed 1 month ago

fortysix2ahead commented 3 months ago

V0.17 introduced the option to skip if there are no changes. It seems that backrest does not have support for that.

To Reproduce Add --skip-if-unchanged to backup options and try to make a snapshot

Expected behavior No snapshot created and no event should be visible.

Screenshots Backrest fails with the following error message:

[restic] command: /bin/restic-0.17.0 snapshots --json -o sftp.args=-oBatchMode=yes --tag plan:playground,created-by:pi5
...
[restic] command: /bin/restic-0.17.0 backup --json /home/playground -o sftp.args=-oBatchMode=yes --exclude-caches --tag plan:playground --tag created-by:pi5 --parent ad9c71eadde1f92dada2388f9da8d461c8502610fc09f23a64d8017d3bd21f20 --host pi5 --skip-if-unchanged
[restic] {"message_type":"summary","files_new":0,"files_changed":0,"files_unmodified":105,"dirs_new":0,"dirs_changed":0,"dirs_unmodified":60,"data_blobs":0,"tree_blobs":0,"data_added":0,"data_added_packed":0,"total_files_processed":105,"total_bytes_processed":1776972,"total_duration":0.016859796}

task "backup for plan \"playground\"" returned error: failed to backup: command "/bin/restic-0.17.0 backup --json /home/playground -o sftp.args=-oBatchMode=yes --exclude-cac..." failed: processing command output: no summary event found
Output:
{"message_type":"summary","files_new":0,"files_changed":0,"files_unmodified":105,"dirs_new":0,"dirs_changed":0,"dirs_unmodified":60,"data_blobs":0,"tree_blobs":0,"data_added":0,"data_added_packed":0,"total_files_processed":105,"total_bytes_processed":1776972,"total_duration":0.016859796}

Platform Info

oliver-la commented 3 months ago

The flag works almost without issues for me, albeit not as one would expect. Snapshots are still created because restic has a weird handling of absolute and relative paths in combination with the flag. As documented here, backrest would need to change the directory first and use relative paths to backup. That might work for single path snapshots, not with multiple paths - at least it seems it'd be hell to implement.

An alternative might be to provide an extra checkbox "Skip identical snapshots" that does a dry-run of the command first. If the output indicates any changes, backrest would run the backup without the dry-run option. Otherwise it'd skip the backup and maybe also provide a visual indication alongside the dry-run command output.

I'm new to restic and backrest, so I'm just spitballing here. Maybe I'm missing something.

To circle back to your issue: Backrest marks snapshots as failed if no snapshot is created using this flag. Otherwise it works as expected.

fortysix2ahead commented 3 months ago

Yes, I am sure. The backrest header says 1.4.0 and the docker image contains restic 0.17.0, which also says 0.17 when called with --version arg.

fortysix2ahead commented 3 months ago

But you are right, actually it works as expected. It's just confusing, that the failed event appears in the list of backups, pretending it being backup that has failed. Pressing "Clear Error History" button removes those events.

garethgeorge commented 3 months ago

Thanks for the report -- overlooked this when initially scanning the update log :)

Interestingly a summary event is definitely printed -- but Backrest ignores it because it fails validation due to the missing snapshot ID.

This probably needs to be a new boolean flag in plan settings. There's some interesting questions of what this should look like in the UI as well i.e. how should all of the backup operations that don't add snapshots be displayed without cluttering the UI (e.g. should they all be linked to the last created snapshot ID?).

Interested to hear what behaviors people would expect.

oliver-la commented 2 months ago

@garethgeorge In my opinion, a skipped snapshot should not appear in the tree view. Thus, I'm able to quickly see a timeline of changes and in case of a restore, be able to pinpoint the precise snapshot I want to restore. To verify the backups are running as scheduled, a corresponding entry in the operation list should be enough. What do you think?

fortysix2ahead commented 2 months ago

I have a similar feeling. Skipped snapshots should not appear. If have a working folder and you worked on a single document in that folder over the last four weeks it would be easy to see, yes there's a change last Friday, last Tuesday and two weeks ago. You could easily search and restore what you want.

patrontheo commented 1 month ago

would love support for that as well :)

garethgeorge commented 1 month ago

Supported in 1.6.0