search

flightstats / hub

fault tolerant, highly available service for data storage and distribution
http://www.flightstats.com
MIT License
103 stars 35 forks source link

Evaluate DynamoDB auto-scaling for hub channels and webhooks #987

Open moulliet opened 6 years ago

moulliet commented 6 years ago

Currently all of the hub tables have hard limits, which can be exceeded if usage patterns change. This could minimize that risk. https://aws.amazon.com/blogs/aws/new-auto-scaling-for-amazon-dynamodb/

moulliet commented 6 years ago

This feature would be a good fit for our prod envs, which have consistent read throughput and do not spike. The cost savings would not be significant, as our total DynamoDB spend for the hub is less than $200/month across all 10 clusters. This feature would not be as good a fit for our dev and staging environments. The regular running of integration tests (which create, read and delete many channels) causes significant load spikes.