search

k1LoW / awspec

RSpec tests for your AWS resources.
MIT License
1.17k stars 193 forks source link

API Throttling - Possible solutions #122

Open igorlg opened 8 years ago

igorlg commented 8 years ago

@k1LoW I've been having some intermittent issues running tests. Ultimately, seems to be that AWS is throttling API calls, which is confirmed by their support...

I was wondering if there's a way to build a mechanism no AWSpec to work around this issue... i.e. handing throttling errors and not failing tests...

I can implement this and submit a pull request, no problem... Just would like to discuss it first.

k1LoW commented 8 years ago

Thank you for good issue.

awspec retry 3 times (aws-sdk default. See https://github.com/aws/aws-sdk-ruby/issues/705 https://github.com/aws/aws-sdk-ruby/blob/282f9719cf7c6389bee935ab17360e6dd29956d3/aws-sdk-core/lib/aws-sdk-core/plugins/retry_errors.rb#L14).

If you want to retry more than 3 times, It is necessary to your pull request :)

Regards

Constantin07 commented 8 years ago

@igorlg Good point! I faced the same issue by running concurrently more than one script against the same AWS account. The solution was to implement what suggests AWS https://docs.aws.amazon.com/general/latest/gr/api-retries.html but it was done in Python.

CarlInglisBJSS commented 7 years ago

I have successfully used https://github.com/NoRedInk/rspec-retry to get round this problem. It adds another layer of retries above that which the CLI does. It's not perfect, in so much as it doesn't do exponential backoff, but it's a quick win for what (for us) is a rare condition.

nictrix commented 6 years ago

@CarlInglisBJSS I tried the same thing when the EFS api was throttling my calls, but still got the same error. Are you doing something in the rspect-retry callback method to refresh or reload the connection?