I would like to provide a high level information on what is API throttling and how can we resolve the same below :
-> Amazon throttles API requests for each AWS account on a per-region basis to help the performance of services to ensure that all calls to the Amazon APIs (whether they originate from an application, calls to a command line interface, or the Amazon console) don't exceed the maximum allowed API request rate.
-> Once a user has crossed the threshold of API calls, they run into 'Rate Exceeded' error message
-> These limits are internal information and is not shared by the backend team and hence I can not share the same with you.
~~~ Steps to resolve the throttling ~~~
This issue can be mitigated by implementing exponential backoffs [1] and/or polling [2] in your configuration code. The idea behind Exponential Backoff is to use progressively longer waits between retries for consecutive error responses. This will decrease the number of API calls made and you will not receive the "Rate Exceeded" error. Examples of how to do so can be found here as well [3]. This will introduce some small delay and randomization of API calls made from your account, reducing rate spikes and the chance of experiencing any throttling events. A description of each can be found below.
-> Polling:
Your application might need to call an API repeatedly to check for an update in status. Before you start polling, give the request time to potentially complete. When you begin polling, use an appropriate sleep interval between successive requests. For best results, use an increasing sleep interval.
Note: This is similar to using random delays as you have used.
-> Calculating the sleep interval(Exponential Backoff):
When you have to poll or retry an API request, we recommend using an exponential backoff algorithm to calculate the sleep interval between API calls. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses.
Also here is an interesting article that I would recommend you to read :
>> https://medium.com/pablo-perez/dealing-with-throttling-in-cloudformation-part-i-56993706c676
>> https://medium.com/pablo-perez/dealing-with-throttling-in-cloudformation-part-ii-ece8f6711b74
In times of heavy development we can have some failures
Throttling: Rate exceeded.Error describing stack marketplace-api-infrastructure-stack error=Throttling: Rate exceededI would like to provide a high level information on what is API throttling and how can we resolve the same below :