enova / sidekiq-rate-limiter

Redis-backed, per-worker rate limits for job processing.
http://rubygems.org/gems/sidekiq-rate-limiter
MIT License
192 stars 43 forks source link

Ability to limit the number of concurrent jobs #41

Open heaven opened 3 years ago

heaven commented 3 years ago

Hi, our use case is that we may have hundreds of Sidekiq workers. Users can add their site URL, we analyze the pages and provide recommendations. The problem is we can unintentionally DDoS websites due to the high number of workers that can run in parallel. We need to limit the number of concurrent jobs, e.g. with the same site_id argument.