search

brennerm / aws-quota-checker

Keep an eye on your AWS quotas before you hit their limits
https://pypi.org/project/aws-quota-checker/
MIT License
84 stars 36 forks source link

Avoid TooManyRequestsException using backoff #36

Open velom opened 2 years ago

velom commented 2 years ago

Adds backoff package to avoid TooManyRequestsException happened with high-rated requests. https://docs.aws.amazon.com/servicequotas/2019-06-24/apireference/API_GetServiceQuota.html Now there are retries:

12-Apr-22 16:57:04 [INFO] aws_quota.prometheus - starting /metrics endpoint on port 8080
12-Apr-22 16:57:04 [INFO] aws_quota.prometheus - collecting checks
12-Apr-22 16:57:22 [INFO] aws_quota.prometheus - collected 746 checks
12-Apr-22 16:57:22 [INFO] aws_quota.prometheus - refreshing limits
12-Apr-22 16:57:39 [INFO] backoff - Backing off _maximum(...) for 0.3s (botocore.errorfactory.TooManyRequestsException: An error occurred (TooManyRequestsException) when calling the GetServiceQuota operation: Rate exceeded)
12-Apr-22 16:57:41 [INFO] backoff - Backing off _maximum(...) for 0.6s (botocore.errorfactory.TooManyRequestsException: An error occurred (TooManyRequestsException) when calling the GetServiceQuota operation: Rate exceeded)

In addition I add traceback to logs when error still happens.

velom commented 2 years ago

@brennerm may you look this PR please?