steamcmd / api

Read-only API interface for steamcmd app_info
https://www.steamcmd.net
MIT License
56 stars 7 forks source link

Incorrect data returned #46

Closed jonakoudijs closed 1 year ago

jonakoudijs commented 1 year ago

The following problem was reported to me directly via Discord:

I don't know if you still maintain this api, I would like to warn you that data from this app is being returned wrong. Check the gid manifest: https://api.steamcmd.net/v1/info/1172470 https://steamdb.info/depot/1172471/manifests/ SteamDB Apex Legends Content (Depot 1172471) · Manifests SteamDB record for depot Apex Legends Content (DepotID or AppID: 1172471)

I have not yet checked it myself but this requires investigation.

Sad-theFaceless commented 1 year ago

I'll add my 2 cents to this

I relied on the API to auto-update a specific steam tool, by looking at the public branch "buildid", but the data returned is outdated (11897995) The actual current buildid, as shown by steamdb.info, is 11948519 (at the time I post this message)

jonakoudijs commented 1 year ago

Thanks for the info, very strange that this not always seem to be up-to-date. I am not available for the upcoming few weeks but I hope to have some room to investigate after that. If you have any suggestions or can debug more, that would be great.

cp-fabian-pittroff commented 1 year ago

Hello @jonakoudijs, Same issue here. I tried out the server with the documented docker compose up command. Had to change some configs to get it to work (needed to remove the web-api command and set a redis password). The local running version seems to work fine.

The initial data is correct, so the steamclient api is not the problem. My guess, there is something wrong with the caching. Could there be an issue that the redis cache is not cleaning up and returning old values? The caching also works for me locally.

Tested with this appid: 2089300

Public API (https://api.steamcmd.net/v1/info/2089300):

Output ```json { "data": { "2089300": { "_change_number": 19885394, "_missing_token": false, "_sha": "5688d311a0c05bf911ee5fb2a0b0b23214afd1e6", "_size": 2026, "appid": "2089300", "common": { "gameid": "2089300", "name": "Icarus Dedicated Server", "parent": "1149460", "releasestate": "released", "section_type": "ownersonly", "type": "Tool" }, "config": { "installdir": "Icarus Dedicated Server", "launch": { "0": { "executable": "IcarusServer.exe" } } }, "depots": { "2089301": { "encryptedmanifests": { "newfrontiers": { "download": "C2915A61B34B868CEEFCA697A31A4A1A", "gid": "7A967156704B416DD79A118EB01C30B1", "size": "06C39039E2728E39B50F47EFF5A16676" }, "nightly": { "download": "10A613392282A0A0A55ADD5F62DD52B9", "gid": "E7CD50610EECA61B12FE6CA4B9CA4568", "size": "EC091D2F5F2BE43179EF7004BE92A3DE" }, "temporary": { "download": "9863E561AD2A53F03686BBD6ADAF4180", "gid": "E893A4029EC5B5C0BDCFB0C74E811335", "size": "097A77AD9460845E96705B6A2344D591" }, "veterans": { "download": "50ADFC0098F6898629587E5AB16DC6FE", "gid": "10DAF4221017078D1981F7C13214F2CA", "size": "C35E109AD0F3F048E4A7479971495AC0" } }, "manifests": { "experimental": { "download": "4886594720", "gid": "4575251987351780002", "size": "14272565436" }, "public": { "download": "4886594720", "gid": "4575251987351780002", "size": "14272565436" } } }, "branches": { "experimental": { "buildid": "11891155", "description": "Predeployment Testing", "timeupdated": "1691728105" }, "newfrontiers": { "buildid": "11925860", "pwdrequired": "1", "timeupdated": "1691802224" }, "nightly": { "buildid": "11932613", "pwdrequired": "1", "timeupdated": "1691952205" }, "public": { "buildid": "11891155", "timeupdated": "1691728116" }, "temporary": { "buildid": "11932134", "pwdrequired": "1", "timeupdated": "1691942001" }, "veterans": { "buildid": "11925860", "pwdrequired": "1", "timeupdated": "1691802214" } } } } }, "status": "success" } ```

Local:

Output ```json { "data": { "2089300": { "_change_number": 20028786, "_missing_token": false, "_sha": "c49dab6e4dc80febcfdf57174ac416c864b17e29", "_size": 2312, "appid": "2089300", "common": { "gameid": "2089300", "name": "Icarus Dedicated Server", "parent": "1149460", "releasestate": "released", "section_type": "ownersonly", "type": "Tool" }, "config": { "installdir": "Icarus Dedicated Server", "launch": { "0": { "executable": "IcarusServer.exe" } } }, "depots": { "2089301": { "encryptedmanifests": { "internaltest": { "download": "76D43C2CB182194182626ACE0C455CA4", "gid": "1E03164C50A20AD0844ADC10631C71F8", "size": "A019B0C41410F81FC0B68C3D5DC35E45" }, "newfrontiers": { "download": "BA98352292C0ADB5C8B53E08D1FBF935", "gid": "97AE0FE16EDE78A242CE4B0843555A20", "size": "814779D159569E29DE917BF822E00C73" }, "nightly": { "download": "BFE181A9912B6FF06105B101A0DF529E", "gid": "AD538C031B59999E79B55D15CC8287C1", "size": "48F01CE964CD23194E81BFF2F77A0DB0" }, "temporary": { "download": "15CF150E173E418B6F07E24E5D12D889", "gid": "FA429BDC5A3A727DA23BE23A16F666DC", "size": "56EC200031D98CF437388BA99CE517C8" }, "veterans": { "download": "CE22874F61A9EFFBD2B0DB18D0F3FEC0", "gid": "A570129D5E1A4F026FD8F088D5FF087C", "size": "C2EF8CBD82CFEC59FE81374A6E93EF53" } }, "manifests": { "experimental": { "download": "4640492320", "gid": "751921785065837685", "size": "15367330840" }, "public": { "download": "5668960464", "gid": "4939968099874347657", "size": "16558537381" } } }, "branches": { "experimental": { "buildid": "12023726", "description": "Predeployment Testing", "timeupdated": "1692893370" }, "internaltest": { "buildid": "12023726", "pwdrequired": "1", "timeupdated": "1692890405" }, "newfrontiers": { "buildid": "12021265", "pwdrequired": "1", "timeupdated": "1692872654" }, "nightly": { "buildid": "12015830", "pwdrequired": "1", "timeupdated": "1692817492" }, "public": { "buildid": "12021265", "timeupdated": "1692876582" }, "temporary": { "buildid": "12024107", "pwdrequired": "1", "timeupdated": "1692892590" }, "veterans": { "buildid": "12021265", "pwdrequired": "1", "timeupdated": "1692872645" } } } } }, "status": "success" } ```
Oldcustard commented 1 year ago

Hi, also having this issue with appid 232250. Specifically, my tool looks at the build id for the "valve_ds" branch which was last updated at time 1690589996 according to the API, but has been updated since then in reality

jonakoudijs commented 1 year ago

I can confirm the issue is that the TTL is either not properly set anymore or gets lost. If I check the TTL on app id 232250 for example in the current live environment the redis-cli output shows no TTL set:

127.0.0.1:6379> ttl 232250
(integer) -1

Will further investigate.

jonakoudijs commented 1 year ago

The cache has been cleared and the keys all seem to be expiring as they are supposed to. I will look into a more permanent fix (if I can find the original issue) later this week.

cp-fabian-pittroff commented 1 year ago

I think it could be these two lines: https://github.com/steamcmd/api/blob/master/src/functions.py#L155-L156

rds.expire(app_id, os.environ["CACHE_EXPIRATION"]) could fail and create the infinite stored key value pair.

I'm not very familiar with python and the redis package, but this might work:

expiration = int(os.environ["CACHE_EXPIRATION"])
rds.set(app_id, data, ex=expiration)
jonakoudijs commented 1 year ago

That might indeed have been the cause. The issue has not re-appeared yet and it still feels strange to me. Nonetheless I think that this change would been an improvement to prevent some edge case problems and at the very least minimise the roundtrip to the Redis server by one.

The latest and just released version 1.13.2 contains your suggestion. Thanks for your help Fabian!