Discussion: Fix CI game checks

[Root-Core]

For context: #140

The CI breaks from time to time, because Steam is blocking the requests, the server is region locked or they are just failing.

To solve the issue, we have some options:

• Query SteamDB / ProtonDB / Algolia in addition
• Add delays
• Use proxies / TOR
• Check added IDs only

---

Quote https://github.com/Open-Wine-Components/umu-protonfixes/pull/140#issuecomment-2366915898:

I'm aware of this issue and to solve it, we'd need to find another public API to validate against.

We could try querying SteamDB with all failed IDs. This could at least reduce the risk of missing IDs.

My idea would be to only check the changed / added IDs though.. we have verified all current IDs. Is there a hook for getting changed / added files, that we could use as an argument for the script?

EDIT: SteamDB's autocompletion could be another vector, as it's less likely to block sequential requests.

POST: "https://94he6yatei-dsn.algolia.net/1/indexes/steamdb/query?x-algolia-agent=SteamDB Autocompletion"

[R1kaB3rN]

I think ProtonDB would be doable and unlike SteamDB, it doesn't sit behind a Cloudflare proxy. For ProtonDB, we have a few options and we can either refer to their public database archive from Github or make a request to https://www.protondb.com/api/v1/reports/summaries/$SteamAppId.json to determine whether it's a valid ID or not. However, getting the data from the source (Steam) should always be preferred and I think we should try to prefer that whenever possible. SteamDB uses SteamKit, but there other ports available. If we go that route though, I think we'd have to authenticate with someone's real Steam credentials when connecting to the Steam network.

[doZennn]

I'm personally not a fan of piggybacking off of undocumented API endpoints from projects like ProtonDB or SteamDB. Especially when they explicitly ask not to do that.

I think we'd have to authenticate with someone's real Steam credentials when connecting to the Steam network.

You can query app info without credentials by logging in as anonymous.

[Root-Core]

I implemented the Steam API on .NET many years ago, we can use an API key without providing explicit credentials. The API key can be stored in the configuration of the GitHub project. So it's not public and can't be abused by third parties.

I don't remember what can be queried though, it's been a while.

We could also just use SteamCMD anonymously and request information. I don't think it's strictly rate limited, but I'm also not sure if we can request all games (e.g. if they're region locked).

---

I'm personally not a fan of piggybacking off of undocumented API endpoints from projects like ProtonDB or SteamDB. Especially when they explicitly ask not to do that.

That's a good point. We can still use ProtonDB - I can't find that they don't allow it - and they also get their info from SteamDB. Or we can use Algolia directly, like ProtonDB does for their search. Not sure about their terms though.

[doZennn]

We could also just use SteamCMD anonymously and request information. I don't think it's strictly rate limited, but I'm also not sure if we can request all games (e.g. if they're region locked).

SteamCMD is overkill IMO. We can query PICS, here's a stripped down POC I made from the python steamkit port, no authentication is needed:

import logging
from steam.client import SteamClient
from steam.core.msg import MsgProto
from steam.enums.emsg import EMsg
from steam.utils.proto import proto_to_dict
import vdf

logging.basicConfig(format="%(asctime)s | %(name)s | %(message)s", level=logging.INFO)
LOG = logging.getLogger('Steam App Check')

class Steam(object):
    def __init__(self):
        self.logged_on_once = False

        self.steam = client = SteamClient()
        client.anonymous_login()

        @client.on("error")
        def handle_error(result):
            LOG.info("Logon result: %s", repr(result))

        @client.on("connected")
        def handle_connected():
            LOG.info("Connected to %s", client.current_server_addr)

        @client.on("channel_secured")
        def send_login():
            if self.logged_on_once and self.steam.relogin_available:
                self.steam.relogin()

        @client.on("disconnected")
        def handle_disconnect():
            LOG.info("Disconnected.")

            if self.logged_on_once:
                LOG.info("Reconnecting...")
                client.reconnect(maxdelay=30)

        @client.on("reconnect")
        def handle_reconnect(delay):
            LOG.info("Reconnect in %ds...", delay)

    def get_product_info(self, appids=[]):
        resp = self.steam.send_job_and_wait(MsgProto(EMsg.ClientPICSProductInfoRequest),
                                           {
                                               'apps': map(lambda x: {'appid': x}, appids),
                                           },
                                           timeout=10
                                           )

        if not resp: return {}

        resp = proto_to_dict(resp)

        for app in resp.get('apps', []):
            app['appinfo'] = vdf.loads(app.pop('buffer')[:-1].decode('utf-8', 'replace'))['appinfo']

        return resp

if __name__ == "__main__":
    worker = Steam()

    # All of the below will work
    appids = [
        10, # Half-Life 1 - a paid game
        20590, # Leage of Legends - a valid app that some may own, but currently not available to purchase
        # Retired apps
        1372880, # The Day Before
        2443720 # Concord
    ]
    print(worker.get_product_info(appids))