ixti / sidekiq-throttled

Concurrency and rate-limit throttling for Sidekiq
MIT License
701 stars 75 forks source link

question: is it expected that throttled jobs run out of order? #122

Closed josh-m-sharpe closed 1 year ago

josh-m-sharpe commented 2 years ago

It seems that if I continue to refresh /sidekiq/queues/high - I see a lot of change. And I took a look at the codez and it looks like the gem might work by:

  1. letting sidekiq pluck jobs off the queue normally
  2. re-enqueue them if the job should be throttled

...which would explain why my queue seems so volatile.

Is it possible to avoid this rotation and work the jobs in the order they were enqueued?

Fwiw, I'm throttling multiple jobs with:

Sidekiq::Throttled::Registry.add(:my_throttle,
  :concurrency => {
    :limit => 1
  },
  :threshold => {
    :limit => 5,
    :period => 1.minute
  }
)
 sidekiq_throttle_as :my_throttle
ixti commented 2 years ago

Hi. You're right, current version rotates the queue. Throttling happens on per-worker class, not queue. There's no way to overcome this right now. But I have this on my list of things to improve – a way to configure throttling on a per-queue basis.