Is your feature request related to a problem? Please describe.
Sometimes ScoutSuite sends too many requests to AWS and they start rate-limiting us:
2024-09-19 11:21:59 LT12115 scout[1334] ERROR emr.py L22: Failed to describe EMR cluster: An error occurred
(ThrottlingException) when calling the DescribeCluster operation (reached max retries: 4): Rate exceeded
It appears that ScoutSuite reacts with a generic failure handler and retries the request immediately, which just makes the problem worse.
Describe the solution you'd like
ScoutSuite should detect rate-limit errors and automatically adjust its request rate accordingly.
Describe alternatives you've considered
ScoutSuite does have command-line options to limit request rates: https://github.com/nccgroup/ScoutSuite/wiki/Handling-Rate-Limiting. However, using them requires the user to cancel the scan and re-run it, making some guess at appropriate request rates. Since ScoutSuite reports success even if there were errors such as this, it's easy for users to miss the errors, think that they had a complete scan, and miss important issues.
Is your feature request related to a problem? Please describe.
Sometimes ScoutSuite sends too many requests to AWS and they start rate-limiting us:
It appears that ScoutSuite reacts with a generic failure handler and retries the request immediately, which just makes the problem worse.
Describe the solution you'd like
ScoutSuite should detect rate-limit errors and automatically adjust its request rate accordingly.
Describe alternatives you've considered
ScoutSuite does have command-line options to limit request rates: https://github.com/nccgroup/ScoutSuite/wiki/Handling-Rate-Limiting. However, using them requires the user to cancel the scan and re-run it, making some guess at appropriate request rates. Since ScoutSuite reports success even if there were errors such as this, it's easy for users to miss the errors, think that they had a complete scan, and miss important issues.