rafaelgomesxyz / esgst

An extension that enhances SteamGifts / SteamTrades.
MIT License
147 stars 23 forks source link

Implement adaptive SteamGifts request limits #1671

Closed at46 closed 3 years ago

at46 commented 4 years ago

Is your enhancement request related to a problem? Please describe. Since 8.8.4 all requests to steamgifts are limited to 1 per second https://github.com/rafaelgssa/esgst/issues/1652#issuecomment-723477077 which was a good start to fulfill the new rate limiting system but makes ESGST feel really slow.

Describe the solution you'd like. Instead of a fix 1 request per second limit I would prefer some variable request limit which starts by e.g. 0,1 seconds per request and increases to something like one request per 2 seconds depending on the number of requests you made in the last minute / hour. That way if you only check a few things on SG it would feel fast again and if you do more stuff it will slow down to respect the rate limits and avoid getting 429 error.

Describe alternatives you've considered. Excluding more settings from the request queue like you did for the enter/leave button would also speed up everything but simultaneously increasing the risk of users hitting the rate limit again. Another alternative would be to allow requests until you hit a certain limit and then wait but this in addition with a user opening more stuff on sg might also lead to hitting the rate limit again.

Additional Context Since there is still stuff which makes unnecessary requests in the current version it's probably better to fix that first before implementing a more complex request queue system.

Eiion commented 4 years ago

I've made a similar proposal (as in variable, adjustable limitations) within the comments of one of my issues yesterday. This is part of his reply: "I could definitely allow users to set a limit in the 0-100 range though, I'll implement that for the next version."

As for "Another alternative would be to allow requests until you hit a certain limit and then wait but this in addition with a user opening more stuff on sg might also lead to hitting the rate limit again." - I mentioned that as well. I see no issue with going fast until the limitation is hit, wait for the next minute to start and continue. While waiting one can e.g. read giveaway descriptions and enter giveaways of already displayed giveaways. But at least there would be something there quickly to "work" with.

rafaelgomesxyz commented 3 years ago

This is in beta. This is how it currently works, but I'm open to change it:

One possible downside of this approach is that it has to check how many requests you've made every 15 seconds, which means reading from the storage, which might lead to a higher CPU usage, although I haven't noticed anything major in my short tests.

at46 commented 3 years ago

When you say it needs to check the previously made requests every 15 seconds do you mean all the time even if I do nothing on sg or only when there is something in the queue?

rafaelgomesxyz commented 3 years ago

It was doing it every 15 seconds all the time, but I just pushed a commit that only does it if there are requests in the queue.