schneems
commented
4 years ago In a slack conversation:
One thing I'm concerned about is making the new rate limit logic the default, rather than making it opt-in. Gem users may be surprised by the change in behavior, especially if they added their own rate limit handling somehow
To mitigate this concern, I'm defaulting the behavior to be "off" by default. The user can either disable the rate throttling logic by following steps outlined in the warning and README, or they can upgrade to version 3.0.0 to opt-in to the new behavior.
This is a continuation of #96 but is done from within the same repo so the tests can run
By default when you get a rate-limit event while using the platform-api the method call will fail. Since there's no feedback built into the client, another call might be attempted (for example if the API is being hit via a background worker). This is bad for Heroku because it means we now are seeing a stream of requests that will never complete (because of the rate limit) and it is bad for the end-user because they have a flurry of errors that are unexpected and unhandled.
This PR builds on top of the project https://github.com/zombocom/rate_throttle_client which was built out of the research from https://github.com/schneems/rate-limit-gcra-client-demo to automatically find a value that the client can sleep for so that it can maximize throughput while still minimizing the number of requests that are rate limited.
At a high-level when the client sees a 429 response from the server, it will sleep and then retry the response. If it gets another 429 then the sleep amount will be multiplicatively increased until it can make a successful request.
As the client is able to make successful requests the amount of sleep time is reduced by a subtractive amount based on the current number of requests allowed (as reported by the server), and it's the current value.
In simulations, over time we end up seeing from 2-3% of requests rate limited.
This PR has been built on the work of other changes added to heroics and platform-api
This PR was paired with @lolaodelola