Fails to identify install folder with partially matching name

[sluedecke]

Related to #86 but probably needing a "different" approach.

GOG Linux native stores games in a folder structure like this:

$ find Brigador\ UpArmored\ Edition -maxdepth 1 -type d
Brigador UpArmored Edition
Brigador UpArmored Edition/game
Brigador UpArmored Edition/support
Brigador UpArmored Edition/docs

game contains the actual game installation where one can find goggame-NUMBER.hashdb, support and docs have some files helping GOG integrate into the Linux desktop. Note that for installed DLC there will be additional goggame-DLCNUMBER.hashdb files.

The base folder (e.g. Brigador UpArmored Edition) always has a file named gameinfo which includes the actual game name. Here is the gameinfo for Brigador:

Brigador: Up-Armored Edition
v1.65
56925
en-US
1356485086
1356485086
55584005483167404

So for games storing their configuration and/or saves within the folder, I propose to look for gameinfo, take the first line and match that against the game name found in the ludusavi.manifest.

[sluedecke]

So it is a different approach, since as far as I understand ludusavi, it takes the manifest file, constructs a list of path candidates per game and checks that against the file system and registry entries (for Windows). Correct?

For games like in this issue ludusavi would need to traverse a and check for gameinfo in each subfolder. This will be "fast" since the gameinfo is always in the /<"base"> folder.

[mtkennerly]

That sounds like a good idea. This could be integrated as part of the InstallDirRanking logic.

For Brigador: Up specifically, I would have expected it to already be able to find the folder. It does a fuzzy match between the game name and folder names, and it looks like the two are close enough to register. Is it not coming up at all?

On Windows, there seem to be goggame-*.info files (where * is a series of numbers) in JSON format containing a name field:

{
    "clientId": "50227812873380374",
    "gameId": "1237807960",
    "language": "English",
    "languages": [
        "en-US"
    ],
    "name": "Dead Cells",
    "playTasks": [
        ...
    ]
}

[sluedecke]

Brigador: I am new to rust, so I might actually miss something here. But after some debugging printouts, I cannot detect ludusavi to check for anything else than Brigador: Up-Armored Edition.

goggame-*.info: on Linux these files are present, too. They reside in the Dead Cells/game/ folder. Since I also have all DLCs for Dead Cells installed, each has it's own goggame-*.info file.

[mtkennerly]

I tried debugging it like this:

diff --git a/src/prelude.rs b/src/prelude.rs
index 0dfe398..441d42e 100644
--- a/src/prelude.rs
+++ b/src/prelude.rs
@@ -589,17 +589,22 @@ impl InstallDirRanking {

                 let mut best: Option<(i64, &String)> = None;
                 'dirs: for expected_dir in expected_install_dirs {
+                    println!("looking for install dir {expected_dir}");
                     let ideal = matcher.fuzzy_match(expected_dir, expected_dir);
                     for actual_dir in &actual_dirs {
                         let score = fuzzy_match(&matcher, expected_dir, actual_dir, &ideal);
                         if let Some(score) = score {
                             if let Some((previous, _)) = best {
                                 if score > previous {
+                                    println!("  subdir '{actual_dir}' -- score {score} beats previous {previous}");
                                     best = Some((score, actual_dir));
                                 }
                             } else {
+                                println!("  subdir '{actual_dir}' -- new score {score}");
                                 best = Some((score, actual_dir));
                             }
+                        } else {
+                            println!("  subdir '{actual_dir}' -- irrelevant");
                         }
                         if score == Some(i64::MAX) {
                             break 'dirs;
@@ -611,6 +616,7 @@ impl InstallDirRanking {
             .collect();

         for (score, name, subdir) in scores {
+            println!("game '{name}' -- selecting subdir '{subdir}' with score {score}");
             self.0
                 .insert((root.clone(), name.to_owned()), (score, subdir.to_owned()));
         }

I don't have the game, so I just made an empty Brigador UpArmored Edition folder in steamapps:

$ cargo run -- backup --preview 'Brigador: Up-Armored Edition'
looking for install dir Brigador
  [...]
  subdir 'Brigador UpArmored Edition' -- irrelevant
  [...]
looking for install dir Brigador: Up-Armored Edition
  [...]
  subdir 'Brigador UpArmored Edition' -- new score 537
  [...]
game 'Brigador: Up-Armored Edition' -- selecting subdir 'Brigador UpArmored Edition' with score 537

You can also modify the fuzzy match unit test like this to check specific cases (Some(_) means that it's at least a partial match, while None means it's irrelevant):

diff --git a/src/prelude.rs b/src/prelude.rs
index 0dfe398..7218272 100644
--- a/src/prelude.rs
+++ b/src/prelude.rs
@@ -1158,6 +1158,7 @@ mod tests {
             ("A Fun Game!", "A Fun Game", Some(219)),
             ("A Funner Game", "A Fun Game", Some(209)),
             ("A Fun Game 2", "A Fun Game", Some(219)),
+            ("Brigador: Up-Armored Edition", "Brigador UpArmored Edition", Some(537)),
         ] {
             assert_eq!(
                 output,

[mtkennerly]

One issue with looking for gameinfo/goggame files is that they go away if you uninstall a game (even if you choose to keep save files), or at least goggame files do on Windows. So we can do it as an additional check, but we'd still need to be able to fall back to the current approach when those files are absent.

[sluedecke]

After some more real life and reading Rust tutorials:

I agree that checking goggame-* files is an extra step to detect game folders and thus possible save games.

Using the log statements from your patch above, the Brigador edition is recognized and --preview correctly finds the files to back up. But running ludusavi itself fails to do so ....

sorry for the lengthy comment

$ cargo run -- backup --preview 'Brigador: Up-Armored Edition'
[...]
[src/prelude.rs:570] root = RootsConfig {
    path: StrictPath {
        raw: "/home/MYUSER/Games",
        basis: None,
    },
    store: Gog,
}
Checking scan_root install_parent /home/MYUSER/Games -> /home/MYUSER/Games
[...]
looking for install dir Brigador: Up-Armored Edition
[...]
  subdir 'Brigador UpArmored Edition' -- new score 537
[...]
game 'Brigador: Up-Armored Edition' -- selecting subdir 'Brigador UpArmored Edition' with score 537
[...]
Scan for backup Brigador: Up-Armored Edition: true
[...]
Checking root /home/MYUSER/Games
Checking file Brigador UpArmored Edition -> <base>/game/settings.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/game/settings.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/settings.json
Checking file Brigador UpArmored Edition -> <base>/game/imgui.ini
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/game/imgui.ini
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/imgui.ini
Checking file Brigador UpArmored Edition -> <base>/game/profile.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/game/profile.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/profile.json
Checking file Brigador UpArmored Edition -> <base>/profile.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/profile.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/profile.json
Checking file Brigador UpArmored Edition -> <base>/settings.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/settings.json
Checking file candidate /home/MYUSER/Games/Brigador UpArmored Edition/game/settings.json
[...]
Brigador: Up-Armored Edition [3.46 KiB]:
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/imgui.ini
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/profile.json
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/settings.json

Overall:
  Games: 1
  Size: 3.46 KiB
  Location: /home/MYUSER/Games/00-savegames/0-ludusavi-backup

ludusavi output:

# start ludusavi, then select "Preview"
$ cargo run
[...]
[src/prelude.rs:570] root = RootsConfig {
    path: StrictPath {
        raw: "/home/MYUSER/Games",
        basis: None,
    },
    store: Gog,
}
Checking scan_root install_parent /home/MYUSER/Games -> /home/MYUSER/Games
[...]
expected_install_dirs for Brigador: Up-Armored Edition: [
    "Brigador",
    "Brigador: Up-Armored Edition",
]
[...]
looking for install dir Brigador: Up-Armored Edition
[...]
  subdir 'Brigador UpArmored Edition' -- new score 537
[...]
game 'Brigador: Up-Armored Edition' -- selecting subdir 'Brigador UpArmored Edition' with score 537
[...]
Scan for backup Brigador: Up-Armored Edition: true
[...]
Checking root /home/MYUSER/Games
Checking file NO_INSTALL_DIR -> <base>/profile.json
Checking file candidate /home/MYUSER/Games/<skip>/profile.json
Checking file candidate /home/MYUSER/Games/<skip>/game/profile.json
Checking file NO_INSTALL_DIR -> <base>/game/imgui.ini
Checking file candidate /home/MYUSER/Games/<skip>/game/imgui.ini
Checking file candidate /home/MYUSER/Games/<skip>/game/game/imgui.ini
Checking file NO_INSTALL_DIR -> <base>/settings.json
Checking file candidate /home/MYUSER/Games/<skip>/game/settings.json
Checking file candidate /home/MYUSER/Games/<skip>/settings.json
Checking file NO_INSTALL_DIR -> <base>/game/profile.json
Checking file candidate /home/MYUSER/Games/<skip>/game/profile.json
Checking file candidate /home/MYUSER/Games/<skip>/game/game/profile.json
Checking file NO_INSTALL_DIR -> <base>/game/settings.json
Checking file candidate /home/MYUSER/Games/<skip>/game/settings.json
Checking file candidate /home/MYUSER/Games/<skip>/game/game/settings.json

I used these debug outputs:

diff --git a/src/prelude.rs b/src/prelude.rs
index b47ea45..38be92e 100644
--- a/src/prelude.rs
+++ b/src/prelude.rs
@@ -567,6 +567,7 @@ impl InstallDirRanking {
     pub fn scan(roots: &[RootsConfig], manifest: &crate::manifest::Manifest, subjects: &[String]) -> Self {
         let mut ranking = Self::default();
         for root in roots {
+            dbg!(root);
             ranking.scan_root(root, manifest, subjects);
         }
         ranking
@@ -591,6 +592,18 @@ impl InstallDirRanking {
             })
             .unwrap_or_default();

+        if root.store == Store::Gog {
+            println!(
+                "Checking scan_root install_parent {} -> {}",
+                install_parent.raw(),
+                install_parent.interpret()
+            );
+            dbg!(&actual_dirs);
+            // SL TODO if Store::Gog and Linux: scan actual_dirs for dir/gameinfo and retrieve game name from first line
+            // SL TODO alternatively scan dir/goggame-*.info + dir/game/goggame-*.info (json) and get game name from json field 'name'
+            // SL NOTE goggame-*.info are from root game if .gameId == .rootGameId
+        }
+
         let scores: Vec<_> = subjects
             .into_par_iter()
             .filter_map(|name| {
@@ -601,20 +614,37 @@ impl InstallDirRanking {
                     .unwrap_or_default();
                 let default_install_dir = name.to_string();
                 let expected_install_dirs = &[manifest_install_dirs, vec![&default_install_dir]].concat();
-
+                // TODO.2022-09-15 SL add goggames-*.info based install dirs into expected_install_dirs
+                let brigador = name == "Brigador: Up-Armored Edition";
+                if root.store == Store::Gog && brigador {
+                    println!("expected_install_dirs for {}: {:#?}", name, expected_install_dirs);
+                }
                 let mut best: Option<(i64, &String)> = None;
                 'dirs: for expected_dir in expected_install_dirs {
+                    if brigador {
+                        println!("looking for install dir {expected_dir}");
+                    }
                     let ideal = matcher.fuzzy_match(expected_dir, expected_dir);
                     for actual_dir in &actual_dirs {
                         let score = fuzzy_match(&matcher, expected_dir, actual_dir, &ideal);
                         if let Some(score) = score {
                             if let Some((previous, _)) = best {
                                 if score > previous {
+                                    if brigador {
+                                        println!("  subdir '{actual_dir}' -- score {score} beats previous {previous}");
+                                    }
                                     best = Some((score, actual_dir));
                                 }
                             } else {
+                                if brigador {
+                                    println!("  subdir '{actual_dir}' -- new score {score}");
+                                }
                                 best = Some((score, actual_dir));
                             }
+                        } else {
+                            if brigador {
+                                println!("  subdir '{actual_dir}' -- irrelevant");
+                            }
                         }
                         if score == Some(i64::MAX) {
                             break 'dirs;
@@ -626,6 +656,9 @@ impl InstallDirRanking {
             .collect();

         for (score, name, subdir) in scores {
+            if name == "Brigador: Up-Armored Edition" {
+                println!("game '{name}' -- selecting subdir '{subdir}' with score {score}");
+            }
             self.0
                 .insert((root.clone(), name.to_owned()), (score, subdir.to_owned()));
         }
@@ -659,6 +692,11 @@ pub fn scan_game_for_backup(

     let mut paths_to_check = std::collections::HashSet::<StrictPath>::new();

+    let brigador = name == "Brigador: Up-Armored Edition";
+    if brigador {
+        println!("Scan for backup {}: {}", name, brigador);
+    }
+
     // Add a dummy root for checking paths without `<root>`.
     let mut roots_to_check: Vec<RootsConfig> = vec![RootsConfig {
         path: StrictPath::new(SKIP.to_string()),
@@ -685,6 +723,9 @@ pub fn scan_game_for_backup(
     }

     for root in roots_to_check {
+        if brigador {
+            println!("Checking root {}", root.path.raw());
+        }
         log::trace!(
             "[{name}] adding candidates from {:?} root: {}",
             root.store,
@@ -699,12 +740,22 @@ pub fn scan_game_for_backup(
             let install_dir = ranking.get(&root, name);

             for raw_path in files.keys() {
+                if brigador {
+                    println!(
+                        "Checking file {} -> {}",
+                        install_dir.to_owned().unwrap_or_else(|| "NO_INSTALL_DIR".to_string()),
+                        raw_path
+                    );
+                }
                 log::trace!("[{name}] parsing candidates from: {}", raw_path);
                 if raw_path.trim().is_empty() {
                     continue;
                 }
                 let candidates = parse_paths(raw_path, &root, &install_dir, steam_id, manifest_dir);
                 for candidate in candidates {
+                    if brigador {
+                        println!("Checking file candidate {}", candidate.raw());
+                    }
                     log::trace!("[{name}] parsed candidate: {}", candidate.raw());
                     if candidate.raw().contains('<') {
                         // This covers `SKIP` and any other unmatched placeholders.

[mtkennerly]

So it works from the CLI, but not the GUI? That's very strange since the ranking flow should be the same for both. I've added some logging in master; could you please try it and upload the log file here?

• Run: RUST_LOG=ludusavi=trace cargo run
• Back up only Brigador (use the three-dot menu next to it)
• Find the log file in the same folder as the config file (Linux: $XDG_CONFIG_HOME/ludusavi or ~/.config/ludusavi)

sorry for the lengthy comment

I appreciate the detail \:D

[sluedecke]

Nope, in the UI Brigador does not show up at all, no 'trace' at all in the logfile. From the command line everything works fine and I can find a backup in the backup folder:

Preview:

$ RUST_LOG=ludusavi=trace cargo run -- backup --preview 'Brigador: Up-Armored Edition' 
    Finished dev [optimized + debuginfo] target(s) in 0.20s
     Running `target/debug/ludusavi backup --preview 'Brigador: Up-Armored Edition'`
Brigador: Up-Armored Edition [3.46 KiB]:
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/imgui.ini
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/profile.json
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/settings.json

Overall:
  Games: 1
  Size: 3.46 KiB
  Location: /home/MYUSER/Games/00-savegames/0-ludusavi-backup

Backup

$ RUST_LOG=ludusavi=trace cargo run -- backup --merge 'Brigador: Up-Armored Edition' 
    Finished dev [optimized + debuginfo] target(s) in 0.20s
     Running `target/debug/ludusavi backup --merge 'Brigador: Up-Armored Edition'`Brigador: Up-Armored Edition [3.46 KiB]:
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/imgui.ini
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/profile.json
  - /home/MYUSER/Games/Brigador UpArmored Edition/game/settings.json

Overall:
  Games: 1
  Size: 3.46 KiB
  Location: /home/MYUSER/Games/00-savegames/0-ludusavi-backup

[mtkennerly]

@sluedecke Could you please upload the full log file, not just the CLI output? I'd like to look for any red flags. If you could provide one log file from a GUI-based backup and one from a CLI-based backup, that would be even better, so I can look at how they're different.

Edit to add:

Nope, in the UI Brigador does not show up at all, no 'trace' at all in the logfile

Just to be clear, are you saying there are no trace-level logs at all when running the GUI? Did you launch the GUI with RUST_LOG=ludusavi=trace cargo run?

[sluedecke]

Nope, in the UI Brigador does not show up at all, no 'trace' at all in the logfile

Just to be clear, are you saying there are no trace-level logs at all when running the GUI? Did you launch the GUI with RUST_LOG=ludusavi=trace cargo run?

I did run the GUI it with RUST_LOG set and yes it generates trace logs. I did not find Brigador in the console output, but it will probably be in the traces. See below!

This is the trace for RUST_LOG=ludusavi=trace cargo run -- backup --preview 'Brigador: Up-Armored Edition':

ludusavi_rCURRENT.log

The traces for RUST_LOG=ludusavi=trace cargo run are > 40 MB:

log.zip

[mtkennerly]

The traces for RUST_LOG=ludusavi=trace cargo run are > 40 MB:

Yeah, doing a full scan for all games is very verbose 😅 Could you please use the three dots next to Brigador to just back up that one game? The GUI logs are so long that it actually rolled over and missed the install dir ranking part.

[sluedecke]

I would love to, but Brigador does not show up in the UI at all :(

So I dug around a bit and with more log messages I see that in prelude::scan_game_for_backup where you loop over roots_to_check and retrieve the install_dir, CLI and UI do not agree on the install_dir.

I use roughly this patch (tag SLX helps me find my messages in the trace):

diff --git a/src/prelude.rs b/src/prelude.rs
index f532a36..fbd835c 100644
--- a/src/prelude.rs
+++ b/src/prelude.rs
@@ -18,6 +18,8 @@ const APP_DIR_NAME: &str = "ludusavi";
 const PORTABLE_FLAG_FILE_NAME: &str = "ludusavi.portable";
 const MIGRATION_FLAG_FILE_NAME: &str = ".flag_migrated_legacy_config";

+pub const TRACE_GAME: &str = "Brigador: Up-Armored Edition";
+
 pub type AnyError = Box<dyn std::error::Error>;

 #[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
@@ -545,7 +547,7 @@ pub fn parse_paths(
         .collect()
 }

-#[derive(Clone, Default)]
+#[derive(Clone, Default, Debug)]
 pub struct InstallDirRanking(std::collections::HashMap<(RootsConfig, String), (i64, String)>);

 impl InstallDirRanking {
@@ -662,7 +664,9 @@ pub fn scan_game_for_backup(
     #[allow(unused_variables)] ignored_registry: &ToggledRegistry,
 ) -> ScanInfo {
     log::trace!("[{name}] beginning scan for backup");
-
+    if name == TRACE_GAME {
+        log::trace!("[{name}] SLX ranking: {ranking:#?}");
+    }
     let mut found_files = std::collections::HashSet::new();
     #[allow(unused_mut)]
     let mut found_registry_keys = std::collections::HashSet::new();
@@ -707,6 +711,14 @@ pub fn scan_game_for_backup(

         if let Some(files) = &game.files {
             let install_dir = ranking.get(&root, name);
+            if name == TRACE_GAME {
+                // SLX Brigador: for UI install_dir is None, but should be something
+                log::trace!(
+                    "[{name}] SLX checking root: {root:#?}, install_dir: {:#?}",
+                    &install_dir
+                );
+                log::trace!("[{name}] SLX ranking is: {ranking:#?}");
+            }

             for raw_path in files.keys() {
                 log::trace!("[{name}] parsing candidates from: {}", raw_path);

CLI tells me:

TRACE [ludusavi::prelude] [Brigador: Up-Armored Edition] SLX checking root: RootsConfig {
    path: StrictPath {
        raw: "/home/MYUSER/Games",
        basis: None,
    },
    store: Gog,
}, install_dir: Some(
    "Brigador UpArmored Edition",
)

UI tells me:

TRACE [ludusavi::prelude] [Brigador: Up-Armored Edition] SLX checking root: RootsConfig {
    path: StrictPath {
        raw: "/home/MYUSER/Games",
        basis: None,
    },
    store: Gog,
}, install_dir: None

[sluedecke]

Since I output the ranking in the logs above, I checked those too. They agree on the install_dir, this is what the CLI logs:

TRACE [ludusavi::prelude] [Brigador: Up-Armored Edition] SLX ranking is: InstallDirRanking(
    {
        (
            RootsConfig {
                path: StrictPath {
                    raw: "/home/MYUSER/Games",
                    basis: None,
                },
                store: Gog,
            },
            "Brigador: Up-Armored Edition",
        ): (
            537,
            "Brigador UpArmored Edition",
        ),
    },
)

UI has rankings for many more games and one for Brigador which is similar to the one found in the client.

[sluedecke]

Heureka, I think I found it! Just sec to confirm it...

[sluedecke]

InstallDirRanking::get fetches a candidate and then does for other in self.0.values(). Once it finds an other with higher confidence, it returns none. Thats where it breaks.

Using the CLI, I filter for Brigador so there is no other and it returns Some correct value.

Using the UI there are many others, in my case it was "World of Goo" so it returns None.

[mtkennerly]

Thank you! That's a great find, and a big oversight on my part 😅 It's supposed to loop through to find a higher score for the same folder, but it's just looking for any higher score. It only happens to work for perfect matches since that short-circuits the rest of the logic. I'll fix this as soon as I can and put out a new release.

[sluedecke]

Any time again, ludusavi finally lets me save my saves easily which I did manually for years - so thank you for that. Also thanks for your patience, it took a while to do the testing.

[sluedecke]

Thank you for the fast fix, it works for me!

[sluedecke]

Though the title speaks about identifying games by their gameinfo, a bug has been found and fixed. Can this issue then be closed?

[mtkennerly]

There could still be value in adding the gameinfo lookup, since the folder-based approach may not work in all cases, but those cases should also hopefully be rare now. Of course, even if we started checking gameinfo, then you get into cases where the name may not match the manifest anyway. If you're not seeing any other cases where Ludusavi fails to find the install directory, then yeah, we can leave out the gameinfo functionality until we know we need it.