nickelser / activejob-traffic_control

Rate limiting/job enabling for ActiveJob using distributed locks in Redis or Memcached.
301 stars 28 forks source link

Usage with sidekiq #21

Open camol opened 2 years ago

camol commented 2 years ago

Can this be used along with Sidekiq?

baka-san commented 2 years ago

I'm also wondering about this. I have Sidkiq configured with ActiveJob.

https://github.com/mperham/sidekiq/wiki/Active-Job

sasharevzin commented 2 years ago

Did any of you check?

baka-san commented 2 years ago

Did any of you check?

I'm using GoodJob now.

https://github.com/bensheldon/good_job

sanifhimani commented 1 year ago

Did any of you check?

I did and it doesn't work.

My setup is basic following this wiki from Sidekiq - https://github.com/mperham/sidekiq/wiki/Active-Job.

sdhull commented 10 months ago

Seems to work fine with Sidekiq, though the logs can be a bit confusing. I just tested a very simple job:

# frozen_string_literal: true

class SleepJob < ApplicationJob
  concurrency 1, drop: true

  def perform
    puts "starting job"
    ApplicationRecord.connection.execute "select pg_sleep(5)"
    puts "ending job"
  end
end

Then I enqueued 5 like so: 5.times { SleepJob.perform_later; sleep 1 }

And you see the log messages:


pid=96952 tid=k9o class=SleepJob jid=7a568d0748d5aa923b08bd67 INFO: start
pid=96952 tid=k9o class=SleepJob jid=7a568d0748d5aa923b08bd67 INFO: Performing SleepJob (Job ID: 9cf9ca14-04b8-4b74-9fbf-e3b4c93784b5) from Sidekiq(default) enqueued at 2023-11-23T02:55:01Z
starting job
pid=96952 tid=jrw class=SleepJob jid=e67080016f12e5fc614d0ee4 INFO: start
pid=96952 tid=jrw class=SleepJob jid=e67080016f12e5fc614d0ee4 INFO: Performing SleepJob (Job ID: 7f0bd20b-7695-4811-b07b-642a2f91a8a3) from Sidekiq(default) enqueued at 2023-11-23T02:55:02Z
pid=96952 tid=jrw class=SleepJob jid=e67080016f12e5fc614d0ee4 INFO: Dropping SleepJob due to concurrency
pid=96952 tid=jrw class=SleepJob jid=e67080016f12e5fc614d0ee4 INFO: Performed SleepJob (Job ID: 7f0bd20b-7695-4811-b07b-642a2f91a8a3) from Sidekiq(default) in 51.48ms
pid=96952 tid=jrw class=SleepJob jid=e67080016f12e5fc614d0ee4 elapsed=0.062 INFO: done
pid=96952 tid=jzg class=SleepJob jid=33e5a55407e36f9d5247c8e5 INFO: start
pid=96952 tid=jzg class=SleepJob jid=33e5a55407e36f9d5247c8e5 INFO: Performing SleepJob (Job ID: ca3826d2-092e-44e2-a340-0a0bcac230ed) from Sidekiq(default) enqueued at 2023-11-23T02:55:03Z
pid=96952 tid=jzg class=SleepJob jid=33e5a55407e36f9d5247c8e5 INFO: Dropping SleepJob due to concurrency
pid=96952 tid=jzg class=SleepJob jid=33e5a55407e36f9d5247c8e5 INFO: Performed SleepJob (Job ID: ca3826d2-092e-44e2-a340-0a0bcac230ed) from Sidekiq(default) in 65.05ms
pid=96952 tid=jzg class=SleepJob jid=33e5a55407e36f9d5247c8e5 elapsed=0.07 INFO: done
pid=96952 tid=hsk class=SleepJob jid=f2fcd489d5259b52caa5635a INFO: start
pid=96952 tid=hsk class=SleepJob jid=f2fcd489d5259b52caa5635a INFO: Performing SleepJob (Job ID: dd88a863-26df-4917-8cf1-b507da3e033c) from Sidekiq(default) enqueued at 2023-11-23T02:55:04Z
pid=96952 tid=hsk class=SleepJob jid=f2fcd489d5259b52caa5635a INFO: Dropping SleepJob due to concurrency
pid=96952 tid=hsk class=SleepJob jid=f2fcd489d5259b52caa5635a INFO: Performed SleepJob (Job ID: dd88a863-26df-4917-8cf1-b507da3e033c) from Sidekiq(default) in 60.67ms
pid=96952 tid=hsk class=SleepJob jid=f2fcd489d5259b52caa5635a elapsed=0.07 INFO: done
pid=96952 tid=jrw class=SleepJob jid=95dd472bc64774b35a771a82 INFO: start
pid=96952 tid=jrw class=SleepJob jid=95dd472bc64774b35a771a82 INFO: Performing SleepJob (Job ID: d500888a-8432-4d70-ba37-d9fbe0af3c61) from Sidekiq(default) enqueued at 2023-11-23T02:55:05Z
pid=96952 tid=jrw class=SleepJob jid=95dd472bc64774b35a771a82 INFO: Dropping SleepJob due to concurrency
pid=96952 tid=jrw class=SleepJob jid=95dd472bc64774b35a771a82 INFO: Performed SleepJob (Job ID: d500888a-8432-4d70-ba37-d9fbe0af3c61) from Sidekiq(default) in 65.01ms
pid=96952 tid=jrw class=SleepJob jid=95dd472bc64774b35a771a82 elapsed=0.074 INFO: done
ending job
pid=96952 tid=k9o class=SleepJob jid=7a568d0748d5aa923b08bd67 INFO: Performed SleepJob (Job ID: 9cf9ca14-04b8-4b74-9fbf-e3b4c93784b5) from Sidekiq(default) in 5284.78ms
pid=96952 tid=k9o class=SleepJob jid=7a568d0748d5aa923b08bd67 elapsed=5.32 INFO: done