Support heroic non gog or epic/legendary games

[megheaiulian]

What's your idea?

As currently implemented in #94 and #141 only gog or legendary launcher games are supported by heroic discovery. Would be nice if ludusavi would automatically discover also custom games installed with Wine and added to heroic launcher.

Looks like this is currently specifically only enabled for those 2 launchers and support for all other games could be enabled by tweaking the code around https://github.com/mtkennerly/ludusavi/blob/master/src/scan/launchers/heroic.rs#L76C1-L79C1

[yoanhg421]

I would like this too. Looks like a modified version of "detect_gog_games" would do the trick. I would help developing this but unfortunately, I don't know anything about Rust.

Support for sideloaded games in heroic would be great since right now is the easiest way to install sideloaded games, and I lost count of how many times I have lost my saves and had to start over again.

[yoanhg421]

@mtkennerly

By the way, I tried messing around with the sideload detection and got it working mostly. you can see the test here https://github.com/yoanhg421/ludusavi/actions and the changes here https://github.com/yoanhg421/ludusavi/commit/02923ec3937f25bc362b76af10332eca4dd97b82

I can open a pull request if you want, I really don't know Rust, just copy pasted your code and modified it a little.

However, it doesn't recognize all the games, since the app_name for sideloaded games is random, it needs to look by title only and heroic doesn't allow special characters in the Game Title.

I can continue messing around with it, but, you probably know a better way to that.

Here are a few edge cases:

• Death's Gambit -> saved as Deaths Gambit in heroic, save can't be found.
• A Hat in Time -> Save Files - saved in the game can't find the save either.

Right now, it recognizes 11 out of 15 games which is pretty good, but not perfect.

[mtkennerly]

@yoanhg421 Thanks for looking into this :) I had just started working on something similar - would you be willing to test a branch when I have one ready?

I opened a ticket with Heroic about the title limitations: https://github.com/Heroic-Games-Launcher/HeroicGamesLauncher/issues/3757 . I don't think there's anything we can do about that in the meantime, but you can always make a custom game in Ludusavi as an alias of the title in Heroic.

A Hat in Time -> Save Files - saved in the game can't find the save either.

Where are the save files on your system, and what config does Heroic have for the folder_name and install.platform?

[yoanhg421]

@mtkennerly I would be happy to test the app, no problem,

I added a comment on the issue you opened with heroic, they also have a title length limit, maybe because folder names have a limitation on some OS.

I usually install the games outside proton, so I have easy access to the install folder. here are some examples of the file. These games are not recognized even tough they don't have complex titles.

What about doing a fuzzy match on the title and grab the key with the closest match, instead of normalizing it? It may be a little slower, but it might give us a better match. What do you think?

Do you think some of those games, is because the platform is windows, but I actually installed the game in the Linux filesystem? Since the save is the game <base>/SaveData I can try and move the installation inside the prefix or reinstall the game, and see if that's the problem for "A Hat in Time"

[yoanhg421]

Ok So after making sure every game had a save file, adding the installs folder as a game root, and adding a few aliases I'm only missing 4 out of 20.

• A hat in time works by setting the install folder as a root.
• Death's gambit works by setting an alias for Deaths Gambit -> also fixed by adding

  static RE_APOSTROPHES: Lazy<Regex> = Lazy::new(|| Regex::new(r#"[']"#).unwrap());
  let normalized = RE_APOSTROPHES.replace_all(&normalized, "");
  assert_eq!("foo bar", normalize_title("  Fo'o  Bar  "));

  The games that can't be found now are

• Desperados 3
• Shadow Tactics: Blades of the Shogun - Aiko's Choice
• Blasphemous 2
• ANNO Mutationem

[mtkennerly]

• For A Hat in Time: We try to use the folder_name as the <base>, but the folder_name is set to a subfolder of the game here (Binaries/Win64). Do you have a /home/deck/Documents/Games/A Hat in Time/HatinTimeGame/SaveData folder?
• For Shovel Knight: Treasure Trove: Not sure about this one. What's the winePrefix set in ${heroic}/GamesConfig/35UXdMybX1GJhfByT5eehV.json ? Ludusavi should be looking for ${winePrefix}/drive_*/users/*/AppData/Roaming/Yacht Club Games/Shovel Knight/*.bin (or ${winePrefix}/pfx/drive_*/...).

Could you also try running RUST_LOG=ludusavi=trace cargo run -- backup --preview "Shovel Knight: Treasure Trove" and upload the log file here?

static RE_APOSTROPHES

Makes sense :+1: ~We can add the apostrophe to RE_SYMBOLS.~ ah, it does need to be separate since replacing it with a space doesn't work.

[mtkennerly]

I've updated master with support for sideloaded and Amazon games. I'm only able to test it on Windows, so any testing for other OSes would a big help!

[yoanhg421]

• For A Hat in Time: We try to use the folder_name as the <base>, but the folder_name is set to a subfolder of the game here (Binaries/Win64). Do you have a /home/deck/Documents/Games/A Hat in Time/HatinTimeGame/SaveData folder?

Yes. I have the /home/deck/Documents/Games/A Hat in Time/HatinTimeGame/SaveData where the save is. by putting the Games folder as one of the roots. it fixes the problem with that game, and others that save to the game base folder.

• For Shovel Knight: Treasure Trove: Not sure about this one. What's the winePrefix set in ${heroic}/GamesConfig/35UXdMybX1GJhfByT5eehV.json ? Ludusavi should be looking for ${winePrefix}/drive_*/users/*/AppData/Roaming/Yacht Club Games/Shovel Knight/*.bin (or ${winePrefix}/pfx/drive_*/...).

Could you also try running RUST_LOG=ludusavi=trace cargo run -- backup --preview "Shovel Knight: Treasure Trove" and upload the log file here?

Sorry about this one, I hadn't run the game yet as it was one of the last games I added. I did make sure to run and save every game now so the 4 games I mentioned above I can't get them to work, I tried setting alias, changing the name in heroic, but can't get them to work for some reason. I can try running the command you sent me on the other games and see what's failing. I'll do that tomorrow; I have the games on the SteamDeck and I need to install Rust and an IDE in order to run tests. I've been using my mac to write the code so far.

static RE_APOSTROPHES

Makes sense 👍 ~We can add the apostrophe to RE_SYMBOLS.~ ah, it does need to be separate since replacing it with a space doesn't work.

Yep 😁

[yoanhg421]

I've updated master with support for sideloaded and Amazon games. I'm only able to test it on Windows, so any testing for other OSes would a big help!

I can run tests on Mac and the SteamDeck linux.

I only have "V Rising" on mac installed thru heroic and it detects it fine so far. However, I noticed the backup preview is very slow at the end of the scan.

This only happens on Mac, It prompted me to access data from other Apps, When I say yes, it takes over 2 minutes. to finish the preview, If I say No it finishes in about 15-20 seconds

[yoanhg421]

https://github.com/mtkennerly/ludusavi/assets/26210577/e603a1a8-3f53-47b1-a992-52a8247334b1

[mtkennerly]

Hmm, there's probably a game with a path that's too broad, which we'd need to fix in the manifest. If you run RUST_LOG=ludusavi=debug cargo run -- backup --preview, it should be the last game that gets logged.

[yoanhg421]

Hmm, there's probably a game with a path that's too broad, which we'd need to fix in the manifest. If you run RUST_LOG=ludusavi=debug cargo run -- backup --preview, it should be the last game that gets logged.

This only happens on Mac, It prompted me to access data from other Apps, When I say yes, it takes over 2 minutes. to finish the preview, If I say No it finishes in about 15-20 seconds I don't think it's a game or manifest issue.

[mtkennerly]

ludusavi_rCURRENT.log

Ah, sorry, the log I'm looking for uses trace level, not debug. Could you run it one more time? The log I'm looking for is [{name}] completed scan for backup.

[yoanhg421]

It's trying to read my Library folder [2024-05-17T04:03:48.050Z] TRACE [ludusavi::path] Glob error 2: /Users/yoanhg/Library/**/PrimeGameDetails.dat | attempting to read `/Users/yoanhg/Library/ContainerManager` resulted in an error: Operation not permitted (os error 1)

[yoanhg421]

It looks like it's more than one file Do you need all 5 of them?

[mtkennerly]

Thanks, that's coming from one of the save paths for Epsilon Jump Prime. I've added an exclusion in the manifest (https://github.com/mtkennerly/ludusavi-manifest/commit/da33bf4ceedf8791c4b539f5d424c5a37d47e63d), so it should be fixed when you download the updated copy.

Do you need all 5 of them?

Nope, just the last one was enough :)

[yoanhg421]

Ok. It finished in 2 seconds 😁

[yoanhg421]

I'll test the other games tomorrow and let you know.

[yoanhg421]

So turns out Aiko's Choice wont properly save on Mac, and I tried running the main branch on the steamdeck but got a compile error. something about "cc" missing

Can you send me the linux artifact from your last push?

I can't download it from the Actions Page

[mtkennerly]

Sure, here you go:

ludusavi-v0.23.0-post.6+55a21f3-linux.zip

If you normally use Ludusavi through Flatpak, don't forget to pass --config ~/.var/app/com.github.mtkennerly.ludusavi/config/ludusavi so the standalone executable knows where to find your config.

[yoanhg421]

Thank you. Here is what I have.

{
    "runner": "sideload",
    "app_name": "5EP35cuu11tMvgcNzUohwp",
    "title": "Desperados III",
    "install": {
        "executable": "/home/deck/Documents/Games/Desperados 3/Desperados III.exe",
        "platform": "Windows",
        "is_dlc": false
    },
    "folder_name": "/home/deck/Documents/Games/Desperados 3",
    "art_cover": "https://cdn2.steamgriddb.com/file/sgdb-cdn/grid/0f816b127200f9af246169ed640caaa9.jpg",
    "is_installed": true,
    "art_square": "https://cdn2.steamgriddb.com/file/sgdb-cdn/grid/0f816b127200f9af246169ed640caaa9.jpg",
    "canRunOffline": true,
    "browserUrl": "",
    "customUserAgent": "",
    "launchFullScreen": false
},

(deck@steamdeck Downloads)$ ./ludusavi backup --preview "Desperados 3"

Overall:
  Games: 0
  Size: 0 B
  Location: /home/deck/ludusavi-backup
(deck@steamdeck Downloads)$ ./ludusavi backup --preview "Desperados III"

Overall:
  Games: 0
  Size: 0 B
  Location: /home/deck/ludusavi-backup

Weirdly enough, I uninstalled the game, re-installed it, and now it get's detected properly. i probably did something wrong, I'll do the same with the other 3 games and see if that's the problem then.

[yoanhg421]

(130)(deck@steamdeck Downloads)$ ./ludusavi backup --preview "Desperados III"
Desperados III [4.61 MiB] [+]:
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/audioSettings.save
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/gameState_00/manualSave/lvl_flashback1_00_medium133604756173318382.save
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/gameplaySettings.save
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/userData.save
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/userData.save.bak
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/drive_c/users/steamuser/AppData/Local/Desperados III/user_steam_117093590311632897/videoSettings.save
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/system.reg
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/user.reg
  - [+] /home/deck/Games/Heroic/Prefixes/default/Desperados III/pfx/userdef.reg

Overall:
  Games: 1 [+1]
  Size: 4.61 MiB
  Location: /home/deck/ludusavi-backup

[mtkennerly]

Weirdly enough, I uninstalled the game, re-installed it, and now it get's detected properly. i probably did something wrong, I'll do the same with the other 3 games and see if that's the problem then.

That's pretty weird. I wonder if something in sideload_apps or GamesConfig is changing between installs?

Might be interesting to compare the log for RUST_LOG=ludusavi=trace cargo run -- backup --preview "Problematic Game" before and after reinstalling.

[yoanhg421]

Well, i tried the same with Blaphemous 2 but even after re-installing the game doesn't work. Now I'm out of ideas.

(deck@steamdeck Downloads)$ RUST_LOG=ludusavi=trace ./ludusavi backup --preview "Blasphemous 2"

Overall:
  Games: 0
  Size: 0 B
  Location: /home/deck/ludusavi-backup

        {
            "runner": "sideload",
            "app_name": "hacXe1vuPaPYKPkdbk16hA",
            "title": "Blasphemous 2",
            "install": {
                "executable": "/home/deck/Documents/Games/Blasphemous 2/Blasphemous 2.exe",
                "platform": "Windows",
                "is_dlc": false
            },
            "folder_name": "/home/deck/Documents/Games/Blasphemous 2",
            "art_cover": "https://cdn2.steamgriddb.com/file/sgdb-cdn/grid/71136483ba006733ac2eaab91fba5cde.png",
            "is_installed": true,
            "art_square": "https://cdn2.steamgriddb.com/file/sgdb-cdn/grid/71136483ba006733ac2eaab91fba5cde.png",
            "canRunOffline": true,
            "browserUrl": "",
            "customUserAgent": "",
            "launchFullScreen": false
        }

{
  "hacXe1vuPaPYKPkdbk16hA": {
    "autoInstallDxvk": true,
    "autoInstallDxvkNvapi": false,
    "autoInstallVkd3d": true,
    "preferSystemLibs": false,
    "enableEsync": true,
    "enableMsync": false,
    "enableFsync": true,
    "nvidiaPrime": false,
    "enviromentOptions": [],
    "wrapperOptions": [],
    "showFps": false,
    "useGameMode": true,
    "battlEyeRuntime": true,
    "eacRuntime": true,
    "language": "",
    "beforeLaunchScriptPath": "",
    "afterLaunchScriptPath": "",
    "wineVersion": {
      "bin": "/home/deck/.var/app/com.heroicgameslauncher.hgl/config/heroic/tools/proton/Proton-GE-Proton9-4/proton",
      "name": "Proton - Proton-GE-Proton9-4",
      "type": "proton"
    },
    "winePrefix": "/home/deck/Games/Heroic/Prefixes/default/Blasphemous 2",
    "wineCrossoverBottle": ""
  },
  "version": "v0",
  "explicit": true
}

[mtkennerly]

For Blasphemous 2, Ludusavi should be looking for this (case-insensitively): /home/deck/Games/Heroic/Prefixes/default/Blasphemous 2/pfx/drive_*/users/*/AppData/LocalLow/The Game Kitchen/Blasphemous 2/Savegames . Does that exist?

[yoanhg421]

this is what i have

/home/deck/Games/Heroic/Prefixes/default/Blasphemous 2/drive_c/users/deck/AppData/LocalLow/The Game Kitchen/Blasphemous 2/SteamUser_76561197960271872/Savegames/

Maybe my copy is out of wack

[mtkennerly]

Aha! I think your copy's fine. I've seen other cases like this where a game may or may not insert a user ID folder in the path, particularly with Steam games depending on whether you run them through Steam or directly through the executable (if I remember right - might be more to it).

I've updated PCGamingWiki: https://www.pcgamingwiki.com/w/index.php?title=Blasphemous_2&diff=1489629&oldid=1461287

Ludusavi does check both drive_*/pfx/users_* and drive_*/users_*, so that's okay that you don't have pfx.

[yoanhg421]

Ok. Still not showing yet, but probably the manifest takes time to get updated.

I'll continue to mess around with Aiko's Choice, now it's the only other game that doesn't show up.

The way things look your sideload integration works great, I appreciate all your help.

I tried to learn Rust but its very complicated, specially the types are so strict. :-P