search

awslabs / amazon-dynamodb-tools

Tools to make effective use of DynamoDB easier.
Apache License 2.0
120 stars 25 forks source link

AWS API rate exceeded - operations failing blindly #32

Open Alfaj0r opened 7 months ago

Alfaj0r commented 7 months ago

I just ran table_tagger.py against a few hundred DynamoDB tables. The console output made me feel like a real DevOps champion, having logs that every single one of my tables was tagged.

However, I reviewed CloudTrail logs and found that my work had not happened as fatefully as I had thought. Almost half of the calls logged some sort of error... image

Drilling down for all the errors, they are all the same: image image

The script needs to be enhanced so that:

  1. it does not log successes for API calls that were not successfully executed
  2. it does not exceed the rate limits for the AWS API
LeeroyHannigan commented 7 months ago

Is your requirement for tagging the tables for granular cost in Cost Explorer, if so, there is no longer a need to do this. I'll update the READ ME later this week.

Alfaj0r commented 7 months ago

Hey @LeeroyHannigan thanks for taking a look. You might be thinking about the available "Resource" extra breakdown in cost-explorer, but that's only good for 2 weeks... image

I want this to run for longer-term analysis too.

Is this what you were thinking about?

LeeroyHannigan commented 7 months ago

Yes exactly what I was thinking about. We need to take in the throttling for tagging resources, once I have some bandwidth I'll take a look.