Closed sluedecke closed 1 year ago
This seems to do the trick (now same behaviour as in execute_backup_as_simple
):
diff --git a/src/scan/layout.rs b/src/scan/layout.rs
index 3b0b5d5..6df619c 100644
--- a/src/scan/layout.rs
+++ b/src/scan/layout.rs
@@ -1132,7 +1132,7 @@ impl GameLayout {
.large_file(true);
'item: for file in &scan.found_files {
- if !backup.includes_file(file.path.render()) {
+ if !backup.includes_file(file.effective().render()) {
log::debug!("[{}] skipped: {}", self.mapping.name, file.path.raw());
continue;
}
resulting file now looks like this:
$ 7z l $BACKUPFOLDER/Druidstone_\ The\ Secret\ of\ the\ Menhir\ Forest/backup-20230530T204938Z.zip
7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)
Scanning the drive for archives:
1 file, 108528 bytes (106 KiB)
Listing archive: Druidstone_ The Secret of the Menhir Forest/backup-20230530T204938Z.zip
--
Path = Druidstone_ The Secret of the Menhir Forest/backup-20230530T204938Z.zip
Type = zip
Physical Size = 108528
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2023-03-28 13:08:16 ..... 981 424 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
2023-03-26 20:22:10 ..... 348 156 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
2023-03-28 13:50:18 ..... 1611 776 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
2023-03-27 08:58:22 ..... 1481 725 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
2023-03-24 00:18:28 ..... 635900 98706 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
2023-03-28 14:26:38 ..... 1733 839 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
2023-03-28 13:48:42 ..... 1581 765 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
2023-03-24 00:18:12 ..... 1247 572 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
2023-03-28 14:21:42 ..... 1668 804 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
2023-03-28 13:08:28 ..... 1495 729 drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
------------------- ----- ------------ ------------ ------------------------
2023-03-28 14:26:38 648045 104496 10 files
Thanks for the detailed report! Unfortunately, it looks like this predates cb6cc654f42d155033094cbceef7e2a99909b770. I added some notes to the changelog about how to clean this up, but it's bad since mapping.yaml looks correct, so we don't have an automatic trigger to replace the faulty backups. I'll probably need to add an option to re-generate the latest backup even if there aren't any changes.
I added a better solution (although it's still not ideal) to make new, corrected backups:
Hm ... a debug option indeed.
Is is feasible to scan backups for missing files on startup (maybe as part as a one time "migration" comparable to a config file migration)?
My backups for Factorio still contain "empty" files backup-20230515T212930Z.zip
and backup-20230515T213036Z.zip
and the mapping.yaml
lists quite some files it expects in the backup. At least the first one could be detected and marked as faulty, subsequent/dependent backups should be faulty too.
- name: backup-20230515T212930Z.zip
when: "2023-05-15T21:29:30.982425443Z"
files:
/GAMESDIR/Factorio/game/config/config.ini:
hash: b0cafdf69369aa63052a5ab4b23f77c06981465b
size: 27854
/GAMESDIR/Factorio/game/saves/_autosave1.zip:
hash: 4ad998e4621bdc28f6b10de31f10ce60e8136b8b
size: 1548743
/GAMESDIR/Factorio/game/saves/_autosave2.zip:
hash: 6a17b91bd687a33fce64287da556223e58187661
size: 28255659
/GAMESDIR/Factorio/game/saves/_autosave3.zip:
hash: fe5b6a87343e6775d3637925a723a357529a88a7
size: 28185827
/GAMESDIR/Factorio/game/saves/atomic 01.zip:
hash: 34298831464ffebdf70845c56dc12a8f6a572965
size: 6176539
/GAMESDIR/Factorio/game/saves/burschi 01.zip:
hash: 65fcc2aa8b4a6cb3139f8dce2fd5350eafdd3e4c
size: 3315709
/GAMESDIR/Factorio/game/saves/fp 2.zip:
hash: 539c245c38de4dcd2054d001abdf8ba940294ba5
size: 1558241
/GAMESDIR/Factorio/game/saves/free 01.zip:
hash: 3df8e15c2abd515cc4854139e24ada325b0a85af
size: 27990345
/GAMESDIR/Factorio/game/saves/free 02.zip:
hash: ea0143dc028aa45dad03c071e18983939dd09e97
size: 28235646
/GAMESDIR/Factorio/game/saves/free 03.zip:
hash: 5f2ec767af301f7db7cbc4bb345cacb49236313a
size: 25999266
/GAMESDIR/Factorio/game/saves/free 04.zip:
hash: d320190447e3fb33d0a6e4d33187d08ab989a0d0
size: 28031365
/GAMESDIR/Factorio/game/saves/free 05.zip:
hash: d20adf42ce740f957c0a036e0337ec61030b71d7
size: 27914667
/GAMESDIR/Factorio/game/saves/head scratching.zip:
hash: dc4bd62e087d65b819bd51aa82bfba1aec842dbc
size: 574041
/GAMESDIR/Factorio/game/saves/second 01.zip:
hash: 070def392e10e6944b432e4fa8cb11f5c46af094
size: 3438286
/home/MYUSER/Games/Factorio/game/config/config.ini: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave1.zip: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave2.zip: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave3.zip: ~
/home/MYUSER/Games/Factorio/game/saves/atomic 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/burschi 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 02.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 03.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 04.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 05.zip: ~
/home/MYUSER/Games/Factorio/game/saves/head scratching.zip: ~
/home/MYUSER/Games/Factorio/game/saves/second 01.zip: ~
registry: ~
- name: backup-20230515T213036Z.zip
when: "2023-05-15T21:30:36.169498772Z"
files:
/GAMESDIR/Factorio/game/saves/_autosave2.zip: ~
/GAMESDIR/Factorio/game/saves/_autosave3.zip: ~
/GAMESDIR/Factorio/game/saves/fp 2.zip: ~
/GAMESDIR/Factorio/game/saves/free 01.zip: ~
/GAMESDIR/Factorio/game/saves/free 03.zip: ~
/GAMESDIR/Factorio/game/saves/free 04.zip: ~
/GAMESDIR/Factorio/game/saves/free 05.zip: ~
/home/MYUSER/Games/Factorio/game/config/config.ini: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave1.zip: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave2.zip: ~
/home/MYUSER/Games/Factorio/game/saves/_autosave3.zip: ~
/home/MYUSER/Games/Factorio/game/saves/atomic 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/burschi 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 01.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 02.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 03.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 04.zip: ~
/home/MYUSER/Games/Factorio/game/saves/free 05.zip: ~
/home/MYUSER/Games/Factorio/game/saves/head scratching.zip: ~
/home/MYUSER/Games/Factorio/game/saves/second 01.zip: ~
registry: ~
Is is feasible to scan backups for missing files on startup (maybe as part as a one time "migration" comparable to a config file migration)?
Ideally, I agree that would be better, but I'm worried about a few potential issues:
Yes, these are many points where things can go wrong. And there seems to be no way to handle all scenarios.
Maybe a pragmatic approach like:
automatic checking for empty zip files once (migration style)
optional backup checking as a new feature which users can invoke either for a single backup OR for all backups (with a proper warning). This could be some kind of dry run in restore.
I wasn't happy with the debug option, so I removed it and implemented your second suggestion. I haven't added an automatic check on startup, at least for now, mainly since it might be slow in some situations.
The validation feature covers both simple/zip and full/differential backups. Right now, it only checks the latest full (and differential, if any) for each game, and it doesn't remove any faulty backups (other than the normal retention settings). I'm open to changing/expanding that in the future, but I think this is a good enough start so that we can get the fix available ASAP.
New button on the restore screen:
If there are problems:
If everything is good:
I like that approach, this way one can find faulty backups. Thanks for adding this one!
One remark:
backups-are-valid =
Your backups are valid.
backups-are-invalid =
These games' backups appear to be invalid.
Do you want to create new full backups for these games?
Again great feature, validation found 5 games with faulty backups, starting in October 2022 ... really an old bug. For one I have some good and then only some bad backups. Since I deleted that game, I cannot do a proper backup anymore and will get a validation error. Any chance this is something one can do in code (deleting the last faulty backup entry)?
I can also edit the mappings.yaml manually for this single game.
since only the latest backup is checked, maybe the messages below should say so, too?
Actually, I forgot that I did this, but it validates whichever backups are selected on the restore screen. So normally that would be the latest, but you can select an older one and validate it too.
Any chance this is something one can do in code (deleting the last faulty backup entry)?
Yes, I can definitely add that. I'd like to add the option to validate all backups, and then it could list all the invalid ones by game and give the option to delete some/all of them. But in the meantime, it's okay to edit mapping.yaml manually.
Ludusavi version
v0.18.2
Operating system
Linux (Steam Deck)
Installation method
Cargo
Description
I want to back up a windows game installed from gog using heroic and my ludusavi config.yaml has some redirects to map between my steam deck and my arch linux machine. I invoke ludusavi directly from the compiled sources as:
cargo run -- backup 'Druidstone: The Secret of the Menhir Forest'
Log output looks fine, but the backup-XXX.zip is empty.
Folders are:
/home/MYUSER/Games/Druidstone
- Game data/home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest
- proton prefix, the savegames are below hereludusavi
config.yaml
has these redirects:Further notes
GAMESDIR
related redirects (GAMESDIR
does not exist).Logs
CLI ouput:
Content of
~/.config/ludusavi/ludusavi_rCURRENT.log
: