We are facing an issue where we need to use the job queue to process various tasks that differ a lot in complexity, and the max concurent jobs is not flexible enough.
Here is an example, suppose you want to process uploaded files, each files can vary in size from 1Kb to 500Mb. We can process only two big files at the same time, but we can bro es 20 small files concurrently.
We would like to limit the maximum concurent jobs depending on the current weight of the job being processed.
jms_job_queue:
queue_options:
max_concurrent_jobs: 10 # This limit applies only to the "foo" queue.
max_concurent_weight: 1000
$job1 = new Job();
$job1->setWeight(300);
// ....
$job2 = new Job();
$job2->setWeight(900);
// ....
$job3 = new Job();
$job3->setWeight(700);
// ....
The example above would run job1, job3 then job2
We can submit a PR, but we would appreciate feedback and suggestions
Hi,
We are facing an issue where we need to use the job queue to process various tasks that differ a lot in complexity, and the
max concurent jobsis not flexible enough.Here is an example, suppose you want to process uploaded files, each files can vary in size from 1Kb to 500Mb. We can process only two big files at the same time, but we can bro es 20 small files concurrently.
We would like to limit the maximum concurent jobs depending on the current
weightof the job being processed.The example above would run job1, job3 then job2
We can submit a PR, but we would appreciate feedback and suggestions