The queue is correctly picked up when we create a group of tasks, but doesn't get picked up when we create a chain of tasks. In the case of chain, the queue in the first task is correctly picked up, but the second task gets assigned to the celery queue.
ch = celery.chain(taskA.si(), taskB.si())
However, when we explicitly set the queue option, everything works correctly:
We have a setup with different workers and tasks being executed in different workers. The queues for the tasks are defined in their signatures:
The queue is correctly picked up when we create a
group
of tasks, but doesn't get picked up when we create achain
of tasks. In the case ofchain
, the queue in the first task is correctly picked up, but the second task gets assigned to thecelery
queue.However, when we explicitly set the
queue
option, everything works correctly:Steps to reproduce
Worker A
Run using:
celery -A worker_a worker -l info --concurrency=4 --queue=worker.a
Worker B
Run using:
celery -A worker_b worker -l info --concurrency=4 --queue=worker.b
Main Program
Run using:
python main.py
Expected behavior
task_on_worker_a
to be executed inWorker A
task_on_worker_b
to be executed inWorker B
Actual behavior
task_on_worker_a
gets executed onWorker A
task_on_worker_b
gets schedule oncelery
queue (which is the default queue)