flightstats / hub

fault tolerant, highly available service for data storage and distribution
http://www.flightstats.com
MIT License
103 stars 35 forks source link

Fix issues with S3Batching from prod rollout #1026

Open moulliet opened 6 years ago

moulliet commented 6 years ago

We need to persist the slow channels, so they do not hold back the processing of other channels during a deploy. Slow channels could use multiple threads on the worker to update multiple minutes concurrently. The RegulatorStrategy needs to have an upper bound, and should move in smaller steps.