schneems
commented
4 years ago All tests pass locally:
$ TEST_APP_NAME=schneems-platform-api-test-app rspec ./specClosed schneems closed 4 years ago
schneems
commented
4 years ago All tests pass locally:
$ TEST_APP_NAME=schneems-platform-api-test-app rspec ./specClosed in favor of #103
By default when you get a rate-limit event while using the platform-api the method call will fail. Since there's no feedback built into the client, another call might be attempted (for example if the API is being hit via a background worker). This is bad for Heroku because it means we now are seeing a stream of requests that will never complete (because of the rate limit) and it is bad for the end-user because they have a flurry of errors that are unexpected and unhandled.
This PR builds on top of the project https://github.com/schneems/rate-limit-gcra-client-demo to automatically find a value that the client can sleep for so that it can maximize throughput while still minimizing the number of requests that are rate limited.
At a high level when it sees a 429 response from the server, it will sleep and then retry the response. If it gets another 429 then the sleep amount will be multiplicatively increased until it can make a successful request.
As the client is able to make successful requests the amount of sleep time is reduced by a subtractive amount based off of the current number of requests allowed (as reported by the server), the amount of time since the last rate limit event, and it's current value.
This logic somewhat mirrors TCP "slow start" behavior (though in reverse).
In simulations, over time we end up seeing from 2-3% of requests rate limited.
This PR has been built on the work of other changes added to heroics:
Discussion
In addition to the implementation, the one last unknown is what the default logging behavior should be. While rate-throttling by default provides a good experience, we need to provide feedback to the user letting them know that it is happening.
This PR was paired with @lolaodelola