search

trek10inc / serverless-secrets

An opinionated tool for safely managing and deploying Serverless projects and their secrets.
Other
165 stars 27 forks source link

Validate secrets step is throttled by AWS when there are many secrets in SSM #40

Open jagregory opened 6 years ago

jagregory commented 6 years ago

If your AWS account has many secrets in SSM, the validate secrets procedure will fail with a Throttled Exception from AWS; this is because the AWS provider is calling ssm.describeParameters for all the parameters in the account, without any filter, and after about 20 page requests it starts being throttled by AWS.

Possible solutions:

  1. When validating secrets, instead of listing all the secrets in the account, go lookup each secret wanted by the app.
  2. Add throttling or retry behaviour to the AWS calls, so if the rate limit is hit the calls will retry.

In the meantime, we've had to disable validation of secrets.

ThrottlingException: Rate exceeded
    at Request.extractError (/Users/jag/dev/trading-stream-consumer/node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.callListeners (/Users/jag/dev/trading-stream-consumer/node_modules/aws-sdk/lib/sequential_executor.js:109:20)
    at Request.emit (/Users/jag/dev/trading-stream-consumer/node_modules/aws-sdk/lib/sequential_executor.js:81:10)