Wait interval optimisation

[mxbi]

Hi there, Amazing program - thank you! I was about to start implementing my own when I came across this.

I was wondering about the reasoning about the default wait_interval choice:

    'Danger Zone': {
        "wait_interval": (list, "3, 5")
    }

Most of the time, these saves are a large number of small files, which means that waiting between each file download causes a 20x slowdown in the overall script performance. Disabling this wait time works great for me on a large library of 150 game saves, and saves a few hours.

Perhaps the better approach would be to have no waiting, and some exponential backoff if an error such as a HTTP 429 is received? (e.g. 0.5 then 1 then 2 seconds). Alternatively, one can use some ratelimiting on the actual Steam APIs, but disabling it when fetching the files from Akamai - they can definitely handle it.

[hhhhhojeihsu]

Hi @mxbi,

Thanks for the feedback and I completely agree with your points. The reason for such a long random wait interval is: I want to absolutely minimize the chance of getting banned by Valve.

And the purpose of this program to me is to run silently on server. So I'm fine with the long waiting time as its running in background unattended.

Right now setting wait_interval to 0 would work. Because even you hit the rate limit, the program will wait and retry for a few times.

Overall I think it's good to have the strategy you mentioned implemented. And let the users to choose and acknowledge the slightly increased risk.

[hhhhhojeihsu]

Hi @mxbi, Your suggestion is integrated into the program as the default behavior in v0.0.68. You no longer need to modify the wait_interval in config file. For now random wait only happens when sending requests to Steam not Akami. Appreciate your suggestion 😄 .