mhenrixon
commented
2 years ago Please checkout v7.1.26 https://github.com/mhenrixon/sidekiq-unique-jobs/releases/tag/v7.1.26 as it has quite a few fixes for until and while executing
Open baka-san opened 2 years ago
mhenrixon
commented
2 years ago Please checkout v7.1.26 https://github.com/mhenrixon/sidekiq-unique-jobs/releases/tag/v7.1.26 as it has quite a few fixes for until and while executing
mhenrixon
commented
2 years ago Also, if you are still having trouble: https://github.com/mhenrixon/sidekiq-unique-jobs/releases/tag/v7.1.27 (especially if you are using redis-namespace).
mario-amazing
commented
1 year ago IMHO, the expected behavior is that a new job can only be created after the previous job is completed, @mhenrixon, am I right? Cuz I can create 2 jobs(1 runinng 1 pending), how can I fix it?
class XlsxAppointmentsUpserterJob
include Sidekiq::Job
sidekiq_options queue: 'exports', retry: 5, lock: :until_and_while_executing, on_conflict: { client: :log, server: :reject }
def perform
require 'pry'; binding.pry
end
endhttps://user-images.githubusercontent.com/9537753/217018449-f9747ebd-0c53-4c29-838e-d446877a80dd.mov
Rails 7 sidekiq (7.0.3) sidekiq-unique-jobs (8.0.0)
mhenrixon
commented
1 year ago Cuz I can create 2 jobs(1 runinng 1 pending), how can I fix it?
Use until_executed
jpriollaud
commented
2 weeks ago This also threw me for quite a while. I expected "new job can only be created after the previous job is completed". This is probably the cause of a lot of confusion and open issues around until_and_while_executing
And the readme appears incorrect: https://github.com/mhenrixon/sidekiq-unique-jobs?tab=readme-ov-file#until-executed
The lock is not removed until the perform is actually completed.
MeterSoft
commented
1 week ago have configuration like this sidekiq_options lock: :until_executed But when i create new job it is able to add it to queue and start multiple times becaue each new job has different digest. My job params is empty and i dont know why digest is different.
So after investigation i found that it use all lock args
[{"job_class"=>"Test::PortfolioJob",
"job_id"=>"26ee041e-d8d9-40e9-bc70-ed0ded2ed3ee",
"provider_job_id"=>nil,
"queue_name"=>"default",
"priority"=>nil,
"arguments"=>[],
"executions"=>0,
"exception_executions"=>{},
"locale"=>"en",
"timezone"=>"UTC",
"enqueued_at"=>"2024-09-04T11:22:16.046505000Z",
"scheduled_at"=>"2024-09-04T11:23:16.046320000Z"}]}That is reason why digest is different and it allow multiple jobs
I can not find way how generate it only based on queue and class name
Describe the bug I'm not sure if this is a bug or if I'm misunderstanding the documentation/usage of
:until_and_while_executing. While:until_and_while_executingproperly locks jobs which are queued but haven't started executing (e.g.perform_in(1.hour, ...)), it does not seem to correctly lock through execution of the job.Expected behavior From reading the documentation, I'd expect
:until_and_while_executingto lock until execution has started (seeuntil_executed) AND through execution of the job (seewhile_executing).Current behavior The first subsequent job will be cancelled correctly, while the next job will not be cancelled, and so on alternating in that manner, i.e.
Rails console
The IDs correspond with the Sidekiq output provided below.
Sidekiq output
Sidekiq panel
Note that complete here is indicated the job is dead, as handled by
sidekiq-status.config/initializers/sidekiq.rb
app/sidekiq/test_job.rb
Additional context Desired usage:
1. Cron schedules a job MyJob every ~10 minutes to update data and do various work.
2. The amount of data handled via MyJob can vary considerably and thus may take over 10 minutes.
3a. In the case that 10 minutes has elapsed and the previous MyJob is complete, MyJob should be run again.
3b. In the case that MyJob had a lot of work to do and still hasn't finished after 10 minutes, the new MyJob should be cancelled and not retried. The same process will repeat every 10 minutes until the original MyJob instance has finished and a new one is then executed.