JustArchiNET / ArchiSteamFarm

C# application with primary purpose of farming Steam cards from multiple accounts simultaneously.
Apache License 2.0
11.09k stars 1.04k forks source link

ASF does not idle cards for accounts with big Steam Community inventories due to a Steam server-side problem #3017

Closed MavericksNightmare closed 11 months ago

MavericksNightmare commented 11 months ago

Checklist

ASF version

Latest stable release

ASF variant

win-x64

Bug description

Short Description ASF cannot be used to farm cards for Steam Accounts with big inventories of about 85k+ gamecards.

Why?

Steam Servers return empty badge overview pages for big accounts. ASF cannot read games to farm from these empty pages and never starts farming

Details ASF determines games to farm from the badge overview page https://steamcommunity.com/my/badges?l=english&p=1. For an account logged into Steam, this badge overview page contains the status of the first 150 bages along with pictures of cards obtained by the user and the numer of cards still missing. For big inventories with tens of thousands of cards from many thousands of badges, constructing a view like this is computationally expensive. Returning it within a few seconds when the badge overview page is queried starts getting a major challenge for Steam Servers. To my experience, starting with a Steam Community Inventory card of maybe about 60k cards of thousands of badges, constructing this view starts getting unreliable. Servers start returning empty results. At this "low" card count of 60k+ retrying this several times may eventually succeed. For inventories with card counts of about 85k+ even retrying it very many times typically does not help any more. Steam servers just cannot construct the badge overview page in time and always return an empty page. I actually have contacted Steam Support on this. They say they do not have a solution for this problem.

ASF uses badge overview page to determine games to farm. The empty page usually returned for accounts with 85k+ inventories causes this action to fail systematically. ASF reacts by displaying error messages that badge overview page cannot be loaded despite of 5 retries and will retry to load badge overview page later. This will of course fail for the same reasons. Therefore ASF can never determine games to farm for accounts with big inventories of 85k+ gamecards. Hence it cannot be used for farming cards on such accounts

Expected behavior

I did not really expect farming to work for accounts with big inventories. I am aware that ASF uses badge overview pages for determing games to farm and Steam servers systematically fail to construct them for big accounts. I would be very happy though if this worked and want to suggest a solution approach for this problem (see below)

Actual behavior

Farming does not work at all for Steam accounts with big inventories. Loading badge overview page fails systematically. Retrying to load the page later does not help. ASF outputs error messages, cannot determine games to farm and therefore never starts farming

Steps to reproduce

Just try to run ASF with an account with sufficiently large Steam Community Inventory. To my experience Steam Servers cannot create badge overview pages for accounts with 85k+ gamecards of 15k+ badges (including foil badges). Just start ASF with a big account like this and look at the error messages displayed when loading badge overview page fails.

Possible reason/solution

Inability to construct badge overview pages is a server-side issue that cannot be addressed directly by ASF. Hence the only option seems to stop using the unreliable badge overview page to determine games to farm for accounts with big inventories.

But how can a list with games to farm be obtained then?

The booster creator pages provides a readily accessible list of all games owned by the account (at Steam crafting boosters requires ownership of a game): https://steamcommunity.com/tradingcards/boostercreator

The following URL returns an html-List of all games the account can receive boosters from: https://steamcommunity.com/my/ajaxgetboostereligibility/

This list / URL is unrelated to Steam Inventory or badges and therefore also works for accounts with big inventories where badge overview pages are unavailable. At Steam, Booster Pack eligibility is obtained once all card drops have been received. A reasonable assumption therefore seems to be that for all games on this list farming is not required any more. Unfortunately Booster Eligibility List is known to be "inaccurate". Games for which a user actually can receive boosters because all card drops have been received may be missing on that list. According to Steam Support, the only reliable way to find out whether an account has booster eligibility is to click on the link "How do I earn card drops?" on the badge page and read what is written there.

So a possible alternative strategy to come up with a list of games to farm for big inventories with unavailable badge overview pages might be:

Start with all games from Booster Creator list and remove all games of Booster Eligibility list from that list. For all remaining games, check out badge pages to find out if there are still card drops to receive and start farming if this is the case.

Can you help us with this bug report?

Somehow, I can test and offer feedback, but can't code

Full log.txt recorded during reproducing the problem

Excerpt from log illustrating the failed attempts to query the badge overview page. Some of the URLs contained in the log actually do not read "https://steamcommunity.com/my/badges?l=english&p=1"  but show my profile URL. I rendered this anonymous by replacing my profile URL by "https://steamcommunity.com/my". Retrying later (not shown in the log any more) results in the same issue and does not solve the problem.

2023-09-30 23:11:31|ArchiSteamFarm-25096|INFO|CMM|Init() Logging in to ISteamUserAuth...
2023-09-30 23:11:31|ArchiSteamFarm-25096|INFO|CMM|Init() Success!
2023-09-30 23:11:33|ArchiSteamFarm-25096|INFO|CMM|IsAnythingToFarm() Checking first badge page...
2023-09-30 23:11:34|ArchiSteamFarm-25096|DEBUG|CMM|InternalRequest() InternalServerError <- GET https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:34|ArchiSteamFarm-25096|WARN|CMM|UrlGetToStream() Request failed after 1 attempts!
2023-09-30 23:11:34|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToStream() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:36|ArchiSteamFarm-25096|DEBUG|CMM|InternalRequest() InternalServerError <- GET https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:36|ArchiSteamFarm-25096|WARN|CMM|UrlGetToStream() Request failed after 1 attempts!
2023-09-30 23:11:36|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToStream() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:38|ArchiSteamFarm-25096|DEBUG|CMM|InternalRequest() InternalServerError <- GET https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:38|ArchiSteamFarm-25096|WARN|CMM|UrlGetToStream() Request failed after 1 attempts!
2023-09-30 23:11:38|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToStream() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:40|ArchiSteamFarm-25096|DEBUG|CMM|InternalRequest() InternalServerError <- GET https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:40|ArchiSteamFarm-25096|WARN|CMM|UrlGetToStream() Request failed after 1 attempts!
2023-09-30 23:11:40|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToStream() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:42|ArchiSteamFarm-25096|DEBUG|CMM|InternalRequest() InternalServerError <- GET https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:42|ArchiSteamFarm-25096|WARN|CMM|UrlGetToStream() Request failed after 1 attempts!
2023-09-30 23:11:42|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToStream() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:42|ArchiSteamFarm-25096|WARN|CMM|UrlGetToHtmlDocument() Request failed after 5 attempts!
2023-09-30 23:11:42|ArchiSteamFarm-25096|DEBUG|CMM|UrlGetToHtmlDocument() Request failing: https://steamcommunity.com/my/badges?l=english&p=1
2023-09-30 23:11:42|ArchiSteamFarm-25096|WARN|CMM|IsAnythingToFarm() Could not get badges' information, we will try again later!
2023-09-30 23:11:52|ArchiSteamFarm-25096|INFO|CMM|Stop() Stopping...
2023-09-30 23:11:52|ArchiSteamFarm-25096|INFO|CMM|OnDisconnected() Disconnected from Steam!

Global ASF.json config file

No response

BotName.json config of all affected bot instances

No response

Additional info

No response

JustArchi commented 11 months ago

Makes total sense to me, since this is going to be experimental I'll likely hide it under bot setting as this method is way less accurate.

On TODO with expectation of rather sooner than later, since this sounds like a nice challenge for me :grin: