Open salmonsteak1 opened 1 week ago
Nothing will be done for the worker that is still working on that job?
Yes, that's correct. duration
is the time we can guarantee the concurrency controls to take effect. After that passes, jobs will be unlocked.
does this mean that this
duration
value inlimits_concurrency
refers to how much time the worker has to process this job?
It doesn't necessarily mean the time a job is running because it might still be waiting to be picked from the queue, for example, if workers are busy with other jobs.
Hey, I'm trying to test out the
duration
parameter for thelimits_concurrency
feature. This is how I'm testing it:I will then run 20 of these jobs, with 10 solid queue workers. I've set
concurrency_maintenance_interval
to 1 secondFrom what I can observe, new jobs are being unblocked every 1 second when the dispatcher detects that a worker is running a job longer than the specified
duration
inlimits_concurrency
.However, this means that eventually, all my workers will be hogged up by such jobs.
I know that it's possible to counter this by setting a job timeout using this approach, but I just wanted to clarify the usage of the
duration
value inlimits_concurrency
.Given this current behavior, does this mean that this
duration
value inlimits_concurrency
refers to how much time the worker has to process this job? If it exceeds this duration, a new job will be unblocked (This is checked by the dispatcher every xxx interval, defined inconcurrency_maintenance_interval
). Nothing will be done for the worker that is still working on that job?