samsondav / rihanna

Rihanna is a high performance postgres-backed job queue for Elixir
MIT License
439 stars 47 forks source link

Feature Request: Rate Limiting #12

Open coderberry opened 6 years ago

coderberry commented 6 years ago

One of the hardest things to manage when dealing with background jobs is rate limiting requests.

Here's the free OSS solution I use currently in ruby: https://github.com/enova/sidekiq-rate-limiter

Here is the page that describes the problem and solution (with Sidekiq enterprise):

https://github.com/mperham/sidekiq/wiki/Ent-Rate-Limiting

Thank you for your contributions to OSS!!!

lpil commented 5 years ago

Does this need to be implemented within Rihanna itself? It would be easy for a job to rate limit itself using something like Hammer (https://github.com/ExHammer/hammer) and re-enqueuing itself later if it has been limited.

I would be concerned about the performance implications of implementing this in core Rihanna for regular non-limited users.

samsondav commented 5 years ago

@lpil to be fair, hammer requires either Redis or mnesia (urgh). Many people who use Rihanna do so explicitly because they don't want to bring on another third party service.

lpil commented 5 years ago

It has a pluggable backend so you could use Postgres or whatever else if you want.