search

aws / chalice

Python Serverless Microframework for AWS
Apache License 2.0
10.67k stars 1.01k forks source link

UpdateDomainName retries too quickly #2043

Open ethanw-tm opened 1 year ago

ethanw-tm commented 1 year ago

During recent Chalice deployments, the following error occurred:

chalice.deploy.deployer.ChaliceDeploymentError: ERROR - While deploying your chalice application, received the following error:

 An error occurred (TooManyRequestsException) when calling the UpdateDomainName
  operation (reached max retries: 2): Too Many Requests

It appears that Chalice includes this error in the retryable errors for the UpdateDomainName action: https://github.com/aws/chalice/blob/master/chalice/awsclient.py#L815

However, it uses the default 5 second retry period: https://github.com/aws/chalice/blob/master/chalice/awsclient.py#L2044

This exceeds the rate limit for UpdateDomainName actions per account, which is 1 per 30 seconds per account: https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-control-service-limits-table Additionally, exponential backoff is not implemented for these retries.

Increasing the wait time for retrying and exponential backoff may prevent this issue.

Cheers!