`appinfo.vdf` Magic Changed on Steam Client Beta (June 2024) - Needs Resolved Upstream in `ValvePython/steam`

[sonic2kk]

Relevant upstream PRs:

• ValvePython/vdf#61
• ValvePython/steam#464 (depends on above VDF PR)

---

Please fill out following when reporting a new bug:

Describe the bug
The upstream magic for appinfo.vdf changed again on the latest Steam Client Beta, meaning parse_appinfo from ValvePython/steam is broken. This happened before (ValvePython/steam#418) and this new issue has already been reported upstream (ValvePython/steam#462).

I am making this issue for awareness for anyone who runs into this, and also for us to track this so that we can update the Steam dependency once a version with a fix is available.

To Reproduce
Steps to reproduce the behavior:

1. Open ProtonUp-Qt while on the latest Steam Client Beta
2. With Steam selected, it will open significantly faster than normal
3. Open the Game List
4. Only Steam Shortcuts will be listed

Expected behavior
ProtonUp-Qt should be able to parse the appinfo.vdf, but cannot pending an upstream fix.

Screenshots
N/A.

Desktop (please complete the following information):

• Platform: PC
• System: Arch Linux
• Version: 2.9.2, but hapepens on any version
• How did you install ProtonUp-Qt?: Discover

Additional context
Took a stab at trying to submit a patch for this upstream, but no guarantees my fix will be the one that gets merged as it is still in progress :smile:

Terminal output
The parse_appinfo gives the following output because it cannot parse the magic in the file.

Error getting ctool map from appinfo.vdf: Invalid magic, got b')DV\x07'
Error updating SteamApp info from appinfo.vdf: Invalid magic, got b')DV\x07'

The fix is not as straightforward as adding the magic to the file though, as we need to skip past some parts of the file. And then there are some issues around parsing the binary VDF information in this file under data.

[sonic2kk]

A small effort by me and a much larger effort by the Protontricks maintainer have created a working implementation however changes are required for both ValvePython/steam and ValvePython/vdf. So we will need to bump both versions once updates are in place.

[sonic2kk]

It seems like the only thing we depend on from the steam dependency is the parse_appinfo function. We could implement this function ourselves in a thirdparty/steam/utils/appcache folder and remove the Steam dependency altogether. This would also allow us to potentially make some performance-speciifc tweaks, something I have been looking at recently but have not made much headway with - the generator is the biggest bottleneck when opening with/selecting the Steam launcher. We can optimise a little bit by searching directly for the Steam Play tools AppID, roughly halving the time it takes iirc, and perhaps further optimisations could be made.

This would allow us to cut one of the larger dependencies we have, the steam dependency is approximately 10mb and it has the following dependencies:

• cachetools (very 60kb, negligable) - Only needed by steam dependency.
• pycrytpodomex (Cryptodome dependency is about 7mb, somewhat significant) - Only needed by steam dependency.
• requests - We use requests anyway, so not relevant
• six - Could not find this package in site-packages, maybe not used anymore?
• vdf - We use vdf anyway, so not relevant

So by implementing the Steam dependency ourselves we can save about 17mb - 10mb from the dependency itself, and 7 from removing the dependency on Cryptodome.

The upstream license of steam is compatible with ProtonUp-Qt's as it is less restrictive, so there shouldn't be any issues adding this ourselves from that standpoint.

---

Pros:

• Saves a nice amount of space, the AppImage could go from 91mb -> 74mb and the Flatpak could go from 108mb -> 91mb.
• We get to make customisations ourselves and generally debloat since we only use a tiny part of a big library - Not saying the steam dependency shouldn't be the size that it is but that we're pulling in a lot for just one function.

Cons:

• We will have to apply fixes ourselves which can be a benefit of upstream takes a while, but a drawback if we make significant changes and have to reconcile them.
• Raises questions about where we should and shouldn't do this, we could be opening a can of worms by doing this for one dependency.

Personally I think we should only consider this if:

1. Upstream does not merge the changes fast enough - unsure how actively maintained it is, there haven't been many commits but maybe that's because it's stable rather than anything else. But if we want to do a release and there is no fix in place it may be something to consider, mainly because we only use this part of the Steam dependency.
2. We think we could really get a lot out of owning this dependency ourselves in terms of optimisations and other patches.