Heroic: Add Amazon Games to Games List

sonic2kk commented 11 months ago

Heroic recently introduced support for Amazon Games, in the v2.9.0 release a few days ago. This PR add support for listing installed Amazon Games in the Heroic Games List.

All games marked as "nile" are the Amazon Games. Internally, Heroic refers to Amazon Games as "Nile" games, pretty clever!

Existing Heroic Implementation

As a quick recap on how the existing GOG, sideload and Legendary games work (#168 has a more in-depth breakdown):

Every Heroic game has a unique identifier string called app_name. GOG, sideload and Legendary have different formats (GOG typically uses numbers, sideload and Legendary typically use an alphanumeric string).
Sideload and GOG games store information in /path/to/heroic/sideload_apps/library.json and /path/to/heroic/gog_store/library.json - These store basic game information such as title and app_name (the internal identifier for games managed by Heroic). It also stores an indicator for whether or not a game is installed (is_installed), but no further information about the installation.
Sideloaded apps are only in the library if they are "Installed" (manually added), so it doesn't differentiate between a sideload library and list of installed sideloaded games (if a sideloaded game is in the library, it has to be installed). So all information we need (apart from Wine information) is stored in the sideload library.json
GOG tracks installed games separately in /path/to/heroic/gog_store/installed.json, and its here that we can get the installation location and so on. We relate the entries in library.json and installed.json by their app_name.
- Some GOG games are not marked as installed in library.json even if they are installed, so we have a utility function to cross-reference the installed.json and verify this.
Legendary games are stored differently again, in /path/to/heroic/legendary/installed.json, and this file gives us almost all the information we need (again, apart from Wine information). There is no library JSON for Legendary games, so this file has game install information too.
Finally, each game type has a JSON configuration file named based on its app_name at /path/to/heroic/GamesConfig/<app_name>.json - This file gives us our Wine information, and we can find the right file because we store app_name.

To generate the list of HeroicGames, we have two loops:

One loop generates the list of GOG and sideloaded games, because the file structures are so similar save for a handful of discrepancies which we allow for with utility functions. Each HeroicGame can also find its own config file at GamesConfig/<app_name>.json. The keys for the most part in the library.json are the same, and as far as I can tell, identical for all game types in GamesConfig/<app_name>.json.
A separate loop is used for Legendary games, since the file structure here is slightly different.

Amazon Games Implementation

The reason I recapped the above is to give context for how Amazon Games stores its files and how this implementation works.

Heroic stores Amazon Games information at /path/to/heroic/store_cache/nile_library.json. This has a slightly different top-level object compared to GOG and sideload library.json (it uses "library": [] instead of "games": []), but apart from that it is extremely close to the GOG and sideload library JSON files.

For this reason, trivial code changes were required to add Amazon Games support. We just needed to add the path to the Amazon Games library file to the store_paths list, check for library as well as games, and that was it for getting games to show up in the list. The existing loop was able to parse 99% of the information from the file, as well as from the GamesConfig since the file structure is the same.

The only change required to the loop was for checking the game install path, as it's stored inside a list in the game JSON object. Once that was done, everything worked as expected, because the rest of the file structure was identical. It seems my efforts of making the Heroic Games support easily extensible were a success :smile:

For context, here is a snippet of an Amazon Games entry in `nile_library.json`:

```json { "app_name": "amzn1.adg.product.9a11bfd3-0659-4109-9f8b-2c4e011455ed", "art_cover": "https://m.media-amazon.com/images/I/61a1yHgOVlL.jpg", "art_square": "https://m.media-amazon.com/images/I/61a1yHgOVlL.jpg", "canRunOffline": true, "install": { "install_path": "/home/emma/Games/Heroic/Cat Quest", "install_size": "268.17 MiB", "version": "25423161-0ec7-4e31-9dc9-efc9c55f4b27", "platform": "Windows" }, "folder_name": "Cat Quest", "is_installed": true, "runner": "nile", "title": "Cat Quest", "description": "Cat Quest is a 2d Open World RPG. Set in the fantastic and wonderful world of cats, play as a catventurer as you explore a massive continent crafted in the unique style of tapestry! Relive the good old days of exploring an overworld map as you raid dungeons for epic loot, complete quests and meet the many furry denizens of this world! Also check out the sequel!", "developer": "The Gentlebros Pte. Ltd.", "is_linux_native": false, "is_mac_native": false }, ```

Using a file in `store_cache`

I was poking around to find where Heroic stores nile games, and the place with a file structure which seemed correct was store_cache. This name and location seemed alarming to me, because it's in a "cache" folder, but it seems to persist and stay up-to-date each time an Amazon game is fetched. There is a nile_store folder but it only has information about the linked Amazon account.

There is a nile_config/nile folder, which notably has library.json and installed.json files. But the structure of these files is entirely different to GOG, sideload, and Legendary. It's a JSON list with no top-level object, and it is entirely unformatted.

An example snippet (formatted for readibility):

```json { "asin": "B09X9SDNPM", "asinVersion": 0, "id": "amzn1.adg.product.9a11bfd3-0659-4109-9f8b-2c4e011455ed", "productDetail": { "details": { "backgroundUrl1": "https://m.media-amazon.com/images/I/51x2b1en8VL.jpg", "backgroundUrl2": "https://m.media-amazon.com/images/I/71PfLYQgoJL.jpg", "developer": "The Gentlebros Pte. Ltd.", "esrbRating": "everyone_10_plus", "gameModes": [ "Single Player" ], "genres": [ "Role-playing (RPG)", "Adventure", "Indie" ], "keywords": [ "fantasy", "role playing", "animal protagonists" ], "legacyProductIds": [], "logoUrl": "https://m.media-amazon.com/images/I/512gYANj1oL.png", "otherDevelopers": [], "pegiRating": "ages_3_and_over", "pgCrownImageUrl": "https://m.media-amazon.com/images/I/51LJ5Ss5TNL.jpg", "publisher": "Plug In Digital", "releaseDate": "2017-08-08T00:00:00Z", "screenshots": [ "https://m.media-amazon.com/images/I/71B-FRFgUfL.jpg", "https://m.media-amazon.com/images/I/71CltfvlKKL.jpg", "https://m.media-amazon.com/images/I/612lUarCU4L.jpg", "https://m.media-amazon.com/images/I/71V3DlaBL+L.jpg", "https://m.media-amazon.com/images/I/41Y6XcaCr4L.jpg", "https://m.media-amazon.com/images/I/6198g5Ok3yL.jpg" ], "shortDescription": "Cat Quest is a 2d Open World RPG. Set in the fantastic and wonderful world of cats, play as a catventurer as you explore a massive continent crafted in the unique style of tapestry! Relive the good old days of exploring an overworld map as you raid dungeons for epic loot, complete quests and meet the many furry denizens of this world! Also check out the sequel!", "trailerImageUrl": "https://m.media-amazon.com/images/I/71LYsPoc+iL.jpg", "uskRating": "NO_RATING", "videos": [ "https://m.media-amazon.com/images/I/F1q7YQXRngL.mp4" ], "websites": { "official": "http://thegentlebros.com/catquest/", "steam": "https://store.steampowered.com/app/593280", "support": null, "gog": "https://www.gog.com/game/cat_quest", "facebook": "https://facebook.com/catquestgame" } }, "iconUrl": "https://m.media-amazon.com/images/I/61a1yHgOVlL.jpg" }, "productLine": "Sonic:Game", "sku": "amzn1.resource.bebffde9-40a6-5a53-9c4b-024a23b41f41", "title": "Cat Quest", "type": "Entitlement", "vendorId": "3238d49f-98d6-478b-b708-195666b4c0c5" } ```

This file does contain some extra information, such as publisher, but is missing some other information, such as installation information. It is about 3x the size of the library.json file (~100k vs ~300k) but that is probably not too big of an issue.

The id in this case refers to the app_name, and the installed.json does list the installed games and path, so it would be possible to parse this file and cross-reference between these two files.

Here is a snippet for the equivalent game above in the `installed.json`:

```json { "id": "amzn1.adg.product.9a11bfd3-0659-4109-9f8b-2c4e011455ed", "version": "25423161-0ec7-4e31-9dc9-efc9c55f4b27", "path": "/home/emma/Games/Heroic/Cat Quest", "size": 281193609 } ```

For simplicity, I chose to use the files in store_cache because they more closely matched the other game types. It would still be possible to create a separate loop to generate a HeroicGame using this file and map the values accordingly, but I am not sure how worthwhile that is. If we absolutely needed anything from this file, we could search for id (since the id in this file is equivalent to HeroicGame.app_name) and get the information that way, but I don't think that's necessary.

Extra Details

Tooltip information is preserved with this implementation, it still correctly displays the Wine/Proton version.
- I am not sure if there are any native Linux Amazon Games, if there are I couldn't find any in my library ("is_linux_native": true didn't have any matches), so I am not sure what would happen in the case of a native game.
As far as I can tell, Amazon Games have no web URL, so there is no double-click functionality for the game title (for GOG games, we open the game page with xdg-open). There are URLs in the file mentioned above but these aren't guaranteed to be there, and we would need to consider which would get priority if they did exist. It is easily something that could go in another PR if desired/requested.
There is a nile_installed_info.json file but this appears to only have information about the last installed game, so it is not that useful to us.

Testing

I tested three Amazon Games: Metal Slug, Clouds & Sheep 2, and Cat Quest. Each of these showed up and showed the correct information.

Changing the Wine version and clicking the refresh button also worked. Heroic defaulted to a non-existent Wine version which I had to change, and ProtonUp-Qt correctly noted that the Wine version did not exist.
I found out that the game install location was not being properly mapped because the Games List was able to tell me that the location did not exist in the tooltip, and it fell back to just the game name.

When no Amazon Games are installed, I did not encounter any issues. However, I have not tested compatibility with Heroic versions that don't have Amazon Games support (< v2.9.0). I did, however, test quickly by hiding/pointing the code to invalid paths for the Amazon Games files, and there were no issues, so if the files don't exist (i.e. we're using an older Heroic version), there shouldn't be any issues.

Small diff but big PR description, though I wanted to provide background on how Amazon Games are stored by Heroic and how I came to this implementation, and also why it works in the context of the existing Heroic implementation, without the need to scroll through all the previous discussion.

If this approach is not desirable and you'd prefer use the other mentioned JSON file, I would still be happy to take a look. I am fairly confident that this approach is solid though, at least for an initial implementation. If problems come up later (missing games/no games show up, etc) then we should be fine.

As usual any feedback is appreciated. Thanks! :-)

sonic2kk commented 11 months ago

Side note: The Flatpak filesystem paths will need updated so it can read store_cache when running with Heroic Flatpak. Other builds such as Flatpak/Package Manager are unaffected because they write to ~/.config, and we have full read access here already :-)

DavidoTek commented 11 months ago

I wasn't aware that HGL now support Amazon Games. Supporting more publishers is great!

Heroic refers to Amazon Games as "Nile" games, pretty clever!

I guess they are both rivers, though quite far away from each other ;)

For simplicity, I chose to use the files in store_cache because they more closely matched the other game types

I guess HGL would always have this cache file, so that should work just fine.

I am not sure if there are any native Linux Amazon Games,

I don't think so.

As far as I can tell, Amazon Games have no web URL

The only URLs I found on Amazon Games were the ones to the official games websites. I haven't checked the Amazon Games Windows App though.

However, I have not tested compatibility with Heroic versions that don't have Amazon Games support (< v2.9.0).

I tested using v2.7.1. Sideloaded apps still worked fine.

Thanks as always!

sonic2kk commented 11 months ago

The only URLs I found on Amazon Games were the ones to the official games websites. I haven't checked the Amazon Games Windows App though.

I don't have a Windows machine to test, I did have a Windows VM through Boxes a while back but not right now. If it is requested we can always look into this further :slightly_smiling_face:

DavidoTek / ProtonUp-Qt