torch-ai / similar-web-js

Provides an api client and types for the SimilarWeb service
1 stars 0 forks source link

Throttling limits preventing test scripting #2

Open lancegliser opened 4 years ago

lancegliser commented 4 years ago

The current 10 requests per second limit end up causing most of the tests to fail. This will only get worse over time as more end points are added. I reached out to Brian Sweeter at SimilarWeb to see if there's perhaps an alternate environment we can hit that isn't throttle as strictly.

Depending on the answer, we could use the onRequest hook in Service to introduce some sort of queuing service that can use promise delay chains to run through them.

lancegliser commented 4 years ago

Suggestion from Jeff Ding, the architect on their side: Use the retry options within Axios. If the response code is 429, retry until it succeeds. We'd want to apply a 100 ms delay between to avoid hammering them.