mtkennerly / ludusavi

Backup tool for PC game saves
MIT License
2.46k stars 53 forks source link

Empty backup due to redirects #213

Closed sluedecke closed 1 year ago

sluedecke commented 1 year ago

Ludusavi version

v0.18.2

Operating system

Linux (Steam Deck)

Installation method

Cargo

Description

I want to back up a windows game installed from gog using heroic and my ludusavi config.yaml has some redirects to map between my steam deck and my arch linux machine. I invoke ludusavi directly from the compiled sources as:

cargo run -- backup 'Druidstone: The Secret of the Menhir Forest'

Log output looks fine, but the backup-XXX.zip is empty.

Folders are:

ludusavi config.yaml has these redirects:

redirects:
  - kind: restore
    source: /home/deck/.local/share/Steam/steamapps/common/Dwarf Fortress
    target: /home/MYUSER/Games/06-itch/dwarf-fortress
  - kind: restore
    source: /run/media/mmcblk0p1/gog
    target: /home/MYUSER/Games
  - kind: restore
    source: /GAMESDIR
    target: /home/MYUSER/Games
  - kind: backup
    source: /home/MYUSER/Games
    target: /GAMESDIR
  - kind: restore
    source: /home/deck
    target: /home/MYUSER

Further notes

Logs

CLI ouput:

$ cargo run -- backup 'Druidstone: The Secret of the Menhir Forest'
    Finished dev [optimized + debuginfo] target(s) in 0.26s
     Running `target/debug/ludusavi backup 'Druidstone: The Secret of the Menhir Forest'`
Are you sure you want to proceed with the backup? New save data will be merged into the target folder:

Are you sure you want to proceed with the backup? New save data will be merged into the target folder:

/home/MYUSER/Games/00-savegames/0-ludusavi-backup yes
Druidstone: The Secret of the Menhir Forest [632.86 KiB] [+]:
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
  - [+] /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
    - Redirecting to: /GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log

Overall:
  Games: 1 [+1]
  Size: 632.86 KiB
  Location: /home/MYUSER/Games/00-savegames/0-ludusavi-backup

Content of ~/.config/ludusavi/ludusavi_rCURRENT.log:

[2023-05-30T20:16:04.790Z] DEBUG [ludusavi] Version: 0.18.2
[2023-05-30T20:16:04.791Z] DEBUG [ludusavi::cli] Config on startup: Config { runtime: Runtime { threads: None }, manifest: ManifestConfig { url: "https://raw.githubusercontent.com/mtkennerly/ludusavi-manifest/master/data/manifest.yaml" }, language: English, theme: Light, roots: [RootsConfig { path: StrictPath { raw: "~/.steam/steam", basis: None }, store: Steam }, RootsConfig { path: StrictPath { raw: "~/Games", basis: None }, store: Gog }, RootsConfig { path: StrictPath { raw: "~/.local/share/proton-pfx/0/pfx", basis: None }, store: OtherWine }, RootsConfig { path: StrictPath { raw: "/home/MYUSER/.config/heroic", basis: None }, store: Heroic }, RootsConfig { path: StrictPath { raw: "/home/MYUSER/Games/01-proton-wine/roki", basis: None }, store: OtherWine }, RootsConfig { path: StrictPath { raw: "/home/MYUSER/Games/01-proton-wine/guacamelee-super-turbo-championship-edition", basis: None }, store: OtherWine }, RootsConfig { path: StrictPath { raw: "/home/MYUSER/.config/lutris", basis: None }, store: Lutris }], redirects: [RedirectConfig { kind: Restore, source: StrictPath { raw: "/home/deck/.local/share/Steam/steamapps/common/Dwarf Fortress", basis: None }, target: StrictPath { raw: "/home/MYUSER/Games/06-itch/dwarf-fortress", basis: None } }, RedirectConfig { kind: Restore, source: StrictPath { raw: "/run/media/mmcblk0p1/gog", basis: None }, target: StrictPath { raw: "/home/MYUSER/Games", basis: None } }, RedirectConfig { kind: Restore, source: StrictPath { raw: "/GAMESDIR", basis: None }, target: StrictPath { raw: "/home/MYUSER/Games", basis: None } }, RedirectConfig { kind: Backup, source: StrictPath { raw: "/home/MYUSER/Games", basis: None }, target: StrictPath { raw: "/GAMESDIR", basis: None } }, RedirectConfig { kind: Restore, source: StrictPath { raw: "/home/deck", basis: None }, target: StrictPath { raw: "/home/MYUSER", basis: None } }], backup: BackupConfig { path: StrictPath { raw: "~/Games/00-savegames/0-ludusavi-backup", basis: None }, ignored_games: {"Transistor", "Darwinia", "Chasm", "Brigador: Up-Armored Edition", "Divinity: Original Sin - Enhanced Edition", "Dota 2", "Mindustry", "Mini Metro", "Steel Storm: Burning Retribution", "MouseCraft", "Fez", "Tower of Time", "The Darkside Detective", "The Pinball Arcade", "X-Morph: Defense", "Train Valley", "Kingdom: Classic", "Baldur's Gate II: Enhanced Edition", "Majesty: The Fantasy Kingdom Sim", "Quake III Arena", "Bastion", "Book of Demons", "Prison Architect", "BioShock Infinite", "Broken Sword: Shadow of the Templars: Director's Cut", "Broken Sword 5: The Serpent's Curse", "Shadowgrounds: Survivor", "Moonlighter", "Shadowgrounds", "World of Goo", "Children of Morta", "Creeper World 3: Arc Eternal", "Oxenfree", "DiRT Rally", "Invisible, Inc.", "Owlboy", "Neverwinter Nights 2", "Kingdom: New Lands", "Deponia", "Icewind Dale: Enhanced Edition", "Overload", "Dex", "Guacamelee! Super Turbo Championship Edition", "Hyper Light Drifter", "Stellaris", "Torchlight II", "Kingdoms and Castles", "Wasteland 2", "Torment: Tides of Numenera", "Hollow Knight", "Voxatron", "Avernum", "Streets of Rogue", "7 Billion Humans", "Cosmic Express", "Risk of Rain", "We Are the Dwarves", "Baldur's Gate: Enhanced Edition", "Warhammer 40,000: Gladius - Relics of War", "Alba: A Wildlife Adventure", "Vampire Survivors", "Mable and the Wood", "Team Fortress 2", "The Darkside Detective: A Fumble in the Dark", "Shadowrun Returns", "Tangledeep", "0 A.D.", "Stardew Valley", "Tyranny", "Fury Unleashed", "Disco Elysium", "Stoneshard", "Botanicula", "Minetest", "Warhammer 40,000: Mechanicus", "Dead Cells"}, filter: BackupFilter { exclude_store_screenshots: false, ignored_paths: [], ignored_registry: [] }, toggled_paths: ToggledPaths({}), toggled_registry: ToggledRegistry({}), sort: Sort { key: Name, reversed: false }, retention: Retention { full: 5, differential: 3 }, format: BackupFormats { chosen: Zip, zip: ZipConfig { compression: Deflate }, compression: Compression { deflate: DeflateCompression { level: 6 }, bzip2: Bzip2Compression { level: 6 }, zstd: ZstdCompression { level: 10 } } } }, restore: RestoreConfig { path: StrictPath { raw: "/home/MYUSER/Games/00-savegames/0-ludusavi-backup", basis: None }, ignored_games: {"Baldur's Gate II: Enhanced Edition", "Streets of Rogue", "Overload", "Baldur's Gate: Enhanced Edition", "7 Billion Humans", "0 A.D.", "Kingdoms and Castles", "Minetest", "The Darkside Detective", "Northgard", "Majesty: The Fantasy Kingdom Sim", "MouseCraft", "Dex", "Broken Sword: Shadow of the Templars: Director's Cut", "Dota 2", "Creeper World 3: Arc Eternal", "Hollow Knight", "Divinity: Original Sin - Enhanced Edition", "Shadowgrounds: Survivor", "Wasteland 2", "Shadow Tactics: Blades of the Shogun", "Moonlighter", "Fez", "Broken Sword 5: The Serpent's Curse", "Cosmic Express", "Neverwinter Nights 2", "DiRT Rally", "Spiritfarer", "The Pinball Arcade", "Steel Storm: Burning Retribution", "Pillars of Eternity", "Tangledeep", "Torment: Tides of Numenera", "Train Valley", "Chasm", "Warhammer 40,000: Mechanicus", "We Are the Dwarves", "Hyper Light Drifter", "Stellaris", "Deponia", "Mini Metro", "The Darkside Detective: A Fumble in the Dark", "Mindustry", "Rise to Ruins", "Warhammer 40,000: Gladius - Relics of War", "Stoneshard", "Icewind Dale: Enhanced Edition", "Botanicula", "Kingdom: Classic", "Planescape: Torment Enhanced Edition", "Stardew Valley", "Tyranny", "Bastion", "BioShock Infinite", "Fury Unleashed", "Invisible, Inc.", "Shadowgrounds", "Oxenfree", "Terraria", "Mable and the Wood", "Risk of Rain", "Team Fortress 2", "World of Goo", "Kingdom: New Lands", "NetHack", "Owlboy", "Tower of Time", "Voxatron", "X-Morph: Defense", "Prison Architect", "Shadowrun Returns", "Darwinia", "Quake III Arena", "Transistor", "Torchlight II"}, sort: Sort { key: Name, reversed: false } }, scan: Scan { show_deselected_games: true, show_unchanged_games: true, show_unscanned_games: true }, cloud: Cloud { remote: None, path: "ludusavi-backup", synchronize: true }, apps: Apps { rclone: App { path: StrictPath { raw: "", basis: None }, arguments: "" } }, custom_games: [] }
[2023-05-30T20:16:04.791Z] DEBUG [ludusavi::cli] Invocation: Backup { preview: false, path: None, force: false, merge: false, no_merge: false, update: false, try_update: false, wine_prefix: None, api: false, sort: None, format: None, compression: None, compression_level: None, full_limit: None, differential_limit: None, cloud_sync: false, no_cloud_sync: false, games: ["Druidstone: The Secret of the Menhir Forest"] }
[2023-05-30T20:16:07.069Z] DEBUG [ludusavi::scan::launchers::generic] ranking installations for Steam: /home/MYUSER/.local/share/Steam
[2023-05-30T20:16:07.069Z] DEBUG [ludusavi::scan::launchers::generic] actual install folders: Vampire Survivors | Team Fortress 2 | Necesse | dota 2 beta | Creeper World 3 | Rise to Ruins | Steam Controller Configs
[2023-05-30T20:16:07.070Z] DEBUG [ludusavi::scan::launchers::generic] ranking installations for Gog: /home/MYUSER/Games
[2023-05-30T20:16:07.070Z] DEBUG [ludusavi::scan::launchers::generic] actual install folders: WINE_SPELUNKY | Streets of Rogue | Brigador UpArmored Edition | Iron Marines | Baldurs Gate Enhanced Edition | MouseCraft | if-on-a-winters-night-four-travelers | Tower of Time | Kingdoms and Castles | Terraria | Blasphemous | Planescape Torment Enhanced Edition | Divinity Original Sin  Enhanced Edition | Alba A Wildlife Adventure | The Witcher 3 Wild Hunt GOTY | 7 Billion Humans | Druidstone | World of Goo | 04-steam-backups | Train Valley 2 | RimWorld | 00-savegames | The Darkside Detective | Train Valley | Iris and the Giant | Metro 2033 Redux | Shadowrun Returns | Broforce | .desktop-menu | Tyranny | Stoneshard | Avernum | Moonlighter | steamwindows | Disco Elysium | Heroic | Baldurs Gate II Enhanced Edition | HellbladeSenuasSacrifice | Dead Cells | 01-proton-wine | 03-documents and mods | Invisible Inc | Chasm | Dwarffortress | Nethack | Torment Tides of Numenera | Dex | 06-itch | Stellaris | Grim Dawn | Tangledeep | Worms Revolution | Go | Prey | 09-antimicrox | Torchlight II | Children of Morta | Rise to Ruins | Overload | Pillars of Eternity | Noita | Hyper Light Drifter | Factorio | Dorfromantik | 05-epic | 02-non-store | The Darkside Detective A Fumble in the Dark | Bastion | Lumencraft | Control | Prison Architect | CHUCHEL | cosmic-express | Cyberpunk 2077 | Northgard
[2023-05-30T20:16:07.070Z] DEBUG [ludusavi::scan::launchers::generic] [Druidstone: The Secret of the Menhir Forest] selecting subdir with score 9223372036854775807: Druidstone
[2023-05-30T20:16:07.070Z] DEBUG [ludusavi::scan::launchers::generic] ranking installations for OtherWine: /home/MYUSER/.local/share/proton-pfx/0/pfx
[2023-05-30T20:16:07.070Z] DEBUG [ludusavi::scan::launchers::generic] actual install folders: dosdevices | drive_c
[2023-05-30T20:16:07.072Z] DEBUG [ludusavi::scan::launchers::generic] ranking installations for OtherWine: /home/MYUSER/Games/01-proton-wine/roki
[2023-05-30T20:16:07.072Z] DEBUG [ludusavi::scan::launchers::generic] actual install folders: dosdevices | GLCache | drive_c
[2023-05-30T20:16:07.072Z] DEBUG [ludusavi::scan::launchers::generic] ranking installations for OtherWine: /home/MYUSER/Games/01-proton-wine/guacamelee-super-turbo-championship-edition
[2023-05-30T20:16:07.072Z] DEBUG [ludusavi::scan::launchers::generic] actual install folders: dosdevices | extras | drive_c
[2023-05-30T20:16:07.072Z] INFO [ludusavi::cli] beginning backup with 1 steps
[2023-05-30T20:16:07.074Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
[2023-05-30T20:16:07.074Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
[2023-05-30T20:16:07.074Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
[2023-05-30T20:16:07.074Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
[2023-05-30T20:16:07.074Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
[2023-05-30T20:16:07.076Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
[2023-05-30T20:16:07.076Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
[2023-05-30T20:16:07.076Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
[2023-05-30T20:16:07.076Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
[2023-05-30T20:16:07.076Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
[2023-05-30T20:16:07.077Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
[2023-05-30T20:16:07.077Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
[2023-05-30T20:16:07.078Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
[2023-05-30T20:16:07.078Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
[2023-05-30T20:16:07.078Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
[2023-05-30T20:16:07.078Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
[2023-05-30T20:16:07.078Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
[2023-05-30T20:16:07.079Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
[2023-05-30T20:16:07.079Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
[2023-05-30T20:16:07.079Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
[2023-05-30T20:16:07.079Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
[2023-05-30T20:16:07.079Z] DEBUG [ludusavi::scan] [Druidstone: The Secret of the Menhir Forest] found: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
[2023-05-30T20:16:07.080Z] INFO [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] creating a Full backup: backup-20230530T201607Z.zip
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] skipped: /home/MYUSER/Games/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] Excess backups: []
[2023-05-30T20:16:07.080Z] DEBUG [ludusavi::scan::layout] [Druidstone: The Secret of the Menhir Forest] Removing irrelevant parent: /home/MYUSER/Games/00-savegames/0-ludusavi-backup/Druidstone_ The Secret of the Menhir Forest/registry.yaml
[2023-05-30T20:16:07.080Z] INFO [ludusavi::cli] completed backup
sluedecke commented 1 year ago

This seems to do the trick (now same behaviour as in execute_backup_as_simple):

diff --git a/src/scan/layout.rs b/src/scan/layout.rs
index 3b0b5d5..6df619c 100644
--- a/src/scan/layout.rs
+++ b/src/scan/layout.rs
@@ -1132,7 +1132,7 @@ impl GameLayout {
             .large_file(true);

         'item: for file in &scan.found_files {
-            if !backup.includes_file(file.path.render()) {
+            if !backup.includes_file(file.effective().render()) {
                 log::debug!("[{}] skipped: {}", self.mapping.name, file.path.raw());
                 continue;
             }

resulting file now looks like this:

$ 7z l $BACKUPFOLDER/Druidstone_\ The\ Secret\ of\ the\ Menhir\ Forest/backup-20230530T204938Z.zip 

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 108528 bytes (106 KiB)

Listing archive: Druidstone_ The Secret of the Menhir Forest/backup-20230530T204938Z.zip

--
Path = Druidstone_ The Secret of the Menhir Forest/backup-20230530T204938Z.zip
Type = zip
Physical Size = 108528

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2023-03-28 13:08:16 .....          981          424  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/config.lua
2023-03-26 20:22:10 .....          348          156  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/shinobi.log
2023-03-28 13:50:18 .....         1611          776  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame6.sav
2023-03-27 08:58:22 .....         1481          725  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame3.sav
2023-03-24 00:18:28 .....       635900        98706  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame2.sav
2023-03-28 14:26:38 .....         1733          839  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame8.sav
2023-03-28 13:48:42 .....         1581          765  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame5.sav
2023-03-24 00:18:12 .....         1247          572  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame1.sav
2023-03-28 14:21:42 .....         1668          804  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame7.sav
2023-03-28 13:08:28 .....         1495          729  drive-0/GAMESDIR/Heroic/Prefixes/Druidstone The Secret of the Menhir Forest/pfx/drive_c/users/steamuser/Documents/Ctrl Alt Ninja/Druidstone/savegame4.sav
------------------- ----- ------------ ------------  ------------------------
2023-03-28 14:26:38             648045       104496  10 files
mtkennerly commented 1 year ago

Thanks for the detailed report! Unfortunately, it looks like this predates cb6cc654f42d155033094cbceef7e2a99909b770. I added some notes to the changelog about how to clean this up, but it's bad since mapping.yaml looks correct, so we don't have an automatic trigger to replace the faulty backups. I'll probably need to add an option to re-generate the latest backup even if there aren't any changes.

mtkennerly commented 1 year ago

I added a better solution (although it's still not ideal) to make new, corrected backups:

image

sluedecke commented 1 year ago

Hm ... a debug option indeed.

Is is feasible to scan backups for missing files on startup (maybe as part as a one time "migration" comparable to a config file migration)?

My backups for Factorio still contain "empty" files backup-20230515T212930Z.zip and backup-20230515T213036Z.zip and the mapping.yaml lists quite some files it expects in the backup. At least the first one could be detected and marked as faulty, subsequent/dependent backups should be faulty too.

      - name: backup-20230515T212930Z.zip
        when: "2023-05-15T21:29:30.982425443Z"
        files:
          /GAMESDIR/Factorio/game/config/config.ini:
            hash: b0cafdf69369aa63052a5ab4b23f77c06981465b
            size: 27854
          /GAMESDIR/Factorio/game/saves/_autosave1.zip:
            hash: 4ad998e4621bdc28f6b10de31f10ce60e8136b8b
            size: 1548743
          /GAMESDIR/Factorio/game/saves/_autosave2.zip:
            hash: 6a17b91bd687a33fce64287da556223e58187661
            size: 28255659
          /GAMESDIR/Factorio/game/saves/_autosave3.zip:
            hash: fe5b6a87343e6775d3637925a723a357529a88a7
            size: 28185827
          /GAMESDIR/Factorio/game/saves/atomic 01.zip:
            hash: 34298831464ffebdf70845c56dc12a8f6a572965
            size: 6176539
          /GAMESDIR/Factorio/game/saves/burschi 01.zip:
            hash: 65fcc2aa8b4a6cb3139f8dce2fd5350eafdd3e4c
            size: 3315709
          /GAMESDIR/Factorio/game/saves/fp 2.zip:
            hash: 539c245c38de4dcd2054d001abdf8ba940294ba5
            size: 1558241
          /GAMESDIR/Factorio/game/saves/free 01.zip:
            hash: 3df8e15c2abd515cc4854139e24ada325b0a85af
            size: 27990345
          /GAMESDIR/Factorio/game/saves/free 02.zip:
            hash: ea0143dc028aa45dad03c071e18983939dd09e97
            size: 28235646
          /GAMESDIR/Factorio/game/saves/free 03.zip:
            hash: 5f2ec767af301f7db7cbc4bb345cacb49236313a
            size: 25999266
          /GAMESDIR/Factorio/game/saves/free 04.zip:
            hash: d320190447e3fb33d0a6e4d33187d08ab989a0d0
            size: 28031365
          /GAMESDIR/Factorio/game/saves/free 05.zip:
            hash: d20adf42ce740f957c0a036e0337ec61030b71d7
            size: 27914667
          /GAMESDIR/Factorio/game/saves/head scratching.zip:
            hash: dc4bd62e087d65b819bd51aa82bfba1aec842dbc
            size: 574041
          /GAMESDIR/Factorio/game/saves/second 01.zip:
            hash: 070def392e10e6944b432e4fa8cb11f5c46af094
            size: 3438286
          /home/MYUSER/Games/Factorio/game/config/config.ini: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave1.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave2.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave3.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/atomic 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/burschi 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 02.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 03.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 04.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 05.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/head scratching.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/second 01.zip: ~
        registry: ~
      - name: backup-20230515T213036Z.zip
        when: "2023-05-15T21:30:36.169498772Z"
        files:
          /GAMESDIR/Factorio/game/saves/_autosave2.zip: ~
          /GAMESDIR/Factorio/game/saves/_autosave3.zip: ~
          /GAMESDIR/Factorio/game/saves/fp 2.zip: ~
          /GAMESDIR/Factorio/game/saves/free 01.zip: ~
          /GAMESDIR/Factorio/game/saves/free 03.zip: ~
          /GAMESDIR/Factorio/game/saves/free 04.zip: ~
          /GAMESDIR/Factorio/game/saves/free 05.zip: ~
          /home/MYUSER/Games/Factorio/game/config/config.ini: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave1.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave2.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/_autosave3.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/atomic 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/burschi 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 01.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 02.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 03.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 04.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/free 05.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/head scratching.zip: ~
          /home/MYUSER/Games/Factorio/game/saves/second 01.zip: ~
        registry: ~
mtkennerly commented 1 year ago

Is is feasible to scan backups for missing files on startup (maybe as part as a one time "migration" comparable to a config file migration)?

Ideally, I agree that would be better, but I'm worried about a few potential issues:

sluedecke commented 1 year ago

Yes, these are many points where things can go wrong. And there seems to be no way to handle all scenarios.

Maybe a pragmatic approach like:

mtkennerly commented 1 year ago

I wasn't happy with the debug option, so I removed it and implemented your second suggestion. I haven't added an automatic check on startup, at least for now, mainly since it might be slow in some situations.

The validation feature covers both simple/zip and full/differential backups. Right now, it only checks the latest full (and differential, if any) for each game, and it doesn't remove any faulty backups (other than the normal retention settings). I'm open to changing/expanding that in the future, but I think this is a good enough start so that we can get the fix available ASAP.

New button on the restore screen:

image

If there are problems:

image

If everything is good:

image

sluedecke commented 1 year ago

I like that approach, this way one can find faulty backups. Thanks for adding this one!

sluedecke commented 1 year ago

One remark:

Again great feature, validation found 5 games with faulty backups, starting in October 2022 ... really an old bug. For one I have some good and then only some bad backups. Since I deleted that game, I cannot do a proper backup anymore and will get a validation error. Any chance this is something one can do in code (deleting the last faulty backup entry)?

I can also edit the mappings.yaml manually for this single game.

mtkennerly commented 1 year ago

since only the latest backup is checked, maybe the messages below should say so, too?

Actually, I forgot that I did this, but it validates whichever backups are selected on the restore screen. So normally that would be the latest, but you can select an older one and validate it too.

Any chance this is something one can do in code (deleting the last faulty backup entry)?

Yes, I can definitely add that. I'd like to add the option to validate all backups, and then it could list all the invalid ones by game and give the option to delete some/all of them. But in the meantime, it's okay to edit mapping.yaml manually.