With auto balance, most workers are assigned to the queue with the most pending jobs, not the one with the highest priority

georgms commented 1 year ago

Horizon Version: v5.10.2
Laravel Version: v9.35.1
PHP Version: 8.1.10
Redis Driver & Version: ext-redis 5.3.7
Database Driver & Version: mysqlnd 81.10

Description:

Laravel Queue works can be assigned to multiple queues and will work on them in the configured priority.
Laravel Horizon also has a queue option where multiple queues can be configured.
My assumption is that, in auto balance mode, Horizon would assign most workers to the highest priority queue if it is not empty in order to empty it as fast as possible.
Instead Horizon will assign most workers to the queue with the most pending jobs, regardless of the queue's priority.

Steps To Reproduce:

In horizon.php configure Horizon like so:

   'defaults' => [
    'supervisor-1' => [
        'connection' => 'redis',
        'queue' => ['default', 'low'],
        'balance' => 'auto',
        'maxProcesses' => 10,
    ],
]

Create 10 jobs on the default queue and 1.000 jobs on the low queue.

Expected:

Horizon assigns most workers to the default queue because it has the higher priority.

Actual:

Horizon assigns most workers to the low queue because it has more pending jobs: grafik

github-actions[bot] commented 1 year ago

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!

florisbosch commented 1 year ago

Hi @georgms,

it seems to me that the current behaviour you are explaining is exactly what the 'auto' mode should do.

See documentation: https://laravel.com/docs/9.x/horizon#balancing-strategies

The auto strategy adjusts the number of worker processes per queue based on the current workload of the queue. For example, if your notifications queue has 1,000 pending jobs while your render queue is empty, Horizon will allocate more workers to your notifications queue until the queue is empty.

I guess the reason behind this is to avoid having jobs which will never be resolved if all processes go to the highest priority queue.

Remember it is for 'balancing' you want to even out the number of processes between all queues. I think what you are looking for is just the default Laravel behaviour. Set balancing to 'false' to let Horizon process the queues by the order you have configured.

bernardwiesner commented 1 year ago

you have to set it to false if you want the default behavior of priority. Note you can't use min-processes and max-processes with the false balance, you can only use processes.

'balance' => 'false',
'processes' => 1

When the balance option is set to false, the default Laravel behavior will be used, wherein queues are processed in the order they are listed in your configuration.

It would be nice though if horizon offered a min-process and max-process based on workload together with the false balance in order to prioritize jobs.

laravel / horizon