Celery 4.4.0, django and flower: Celery worker receives task which is already being executed by other worker. Its not executed but added in processed

[svamol]

I have set up with:

Django: 3.1.3 Django_celery_result: 1.1.0 Celery: 4.4.0 amqp: 2.5.2 Celery Flower: latest CELERY_RESULT_BACKEND= Django python db

I have started 10 different celery workers.

When task is submitted its received by celery workers. One of the worker picks it up and start processing it and other workers also received/accept it which cause increase in process count in celery flower. All these workers keep on retrying leading to huge number of process count. For 3 task the processes count is: 40+.

How can we ensure there is no retry from other workers when once worker is already executing tasks.

This works fine with celery 3.x version executed from manage.py celery worker. But its problem with celery version: 4.4.0.

Please let me know if more details are required.

[thedrow]

This repository is for reporting issues for our (now defunct) website. Please file an issue on the appropriate repository.