ixti / sidekiq-throttled

Concurrency and rate-limit throttling for Sidekiq
MIT License
710 stars 76 forks source link

Concurrency 1 only maintened 15 minutes #51

Closed KevinBerthier closed 6 years ago

KevinBerthier commented 6 years ago

A throttling strategy is set to concurrency 1

Sidekiq::Throttled::Registry.add(:big_parser, concurrency: { limit: 1 })

With multiples workers using this strategy

class UpdateCatalogDailyWorker
  include Sidekiq::Worker
  sidekiq_options backtrace: 4, retry: false
  include Sidekiq::Throttled::Worker
  sidekiq_throttle_as :big_parser

  def perform
  end
end

When I run many jobs the no-concurrency is maintained only for 15 min

ixti commented 6 years ago

default lock ttl is 15 minutes. Increase it to be bigger than your jobs are taking:

Sidekiq::Throttled::Registry.add(:big_parser, concurrency: { limit: 1, ttl: 1.hour.to_i })
erated commented 6 years ago

hey Ixti, from what I noticed, even once the job is done, it will not pull in another job until the ttl is done. Is there a way to lock for either: job is done or 1 hour (whatever is shorter)?

ixti commented 6 years ago

@erated Are you sure you are referring to concurrency throttling? As what you describe is a behaviour of threshold throttling.

ziaulrehman40 commented 5 years ago

@ixti I found this after 2 days of googling and stuff. Please have it somewhere in readme. Thanks

kjvarga commented 5 years ago

This is vital information. Found it after noticing my production jobs were not respecting concurrency. +1 to add to the README.

ixti commented 5 years ago

@kjvarga totally open for PRs on improving documentation