[X] I have included the output of celery -A proj report in the issue.
(if you are not able to do this, then at least specify the Celery version affected).
[X] I have verified that the issue exists against the master branch of Celery.
Execute a task that doesn't complete so quickly that you can issue shutdown of the worker during the execution of the task.
When task finishes, worker shuts down
Check the status of the task from backend: in my case using django-db, its still STARTED. I have a hook for on_success where I write some more details of the task to separate database table and this hook is called.
Expected behavior
After issuing a worker shutdown, task running at that time are completed either successfully or with errors and the corresponding status of the task is written to backend (SUCCESS, FAILURE) and correct counters updated.
Actual behavior
After issuing a worker shutdown, task running at that time are completed either successfully or with errors, but the status of the task in backend is not updated and corresponding counters are not updated. Because I use track_started = True, tasks remain in STARTED state.
Checklist
celery -A proj report
in the issue. (if you are not able to do this, then at least specify the Celery version affected).master
branch of Celery.Versions used
cx_Oracle==6.0.2 Django==1.11.6 celery==4.1.0 django-celery-results==1.0.1
Steps to reproduce
on_success
where I write some more details of the task to separate database table and this hook is called.Expected behavior
After issuing a worker shutdown, task running at that time are completed either successfully or with errors and the corresponding status of the task is written to backend (SUCCESS, FAILURE) and correct counters updated.
Actual behavior
After issuing a worker shutdown, task running at that time are completed either successfully or with errors, but the status of the task in backend is not updated and corresponding counters are not updated. Because I use
track_started = True
, tasks remain in STARTED state.Log from Celery worker
[2018-02-20 08:15:39,552: DEBUG/MainProcess] | Consumer: Closing event loop... [2018-02-20 08:15:39,552: DEBUG/MainProcess] | Consumer: Stopping event loop... [2018-02-20 08:15:39,552: DEBUG/MainProcess] | Consumer: Stopping Gossip... [2018-02-20 08:15:39,561: DEBUG/MainProcess] Closed channel #3 [2018-02-20 08:15:39,561: DEBUG/MainProcess] | Consumer: Stopping Heart... [2018-02-20 08:15:39,561: DEBUG/MainProcess] | Consumer: Stopping Control... [2018-02-20 08:15:39,563: DEBUG/MainProcess] Closed channel #2 [2018-02-20 08:15:39,563: DEBUG/MainProcess] | Consumer: Stopping Tasks... [2018-02-20 08:15:39,563: DEBUG/MainProcess] Canceling task consumer... [2018-02-20 08:15:39,564: DEBUG/MainProcess] | Consumer: Stopping Mingle... [2018-02-20 08:15:39,564: DEBUG/MainProcess] | Consumer: Stopping Events... [2018-02-20 08:15:39,564: DEBUG/MainProcess] | Consumer: Stopping Connection... [2018-02-20 08:15:39,564: DEBUG/MainProcess] | Worker: Stopping Pool... [2018-02-20 08:15:48,663: INFO/ForkPoolWorker-17] puppet.run[ce6980ce-7e3a-4aa5-bc25-1891e262d065]: Running Puppet on foo.test.ee using timeout 1 minutes as user erkib [2018-02-20 08:16:48,791: INFO/ForkPoolWorker-17] Task puppet.run[ce6980ce-7e3a-4aa5-bc25-1891e262d065] succeeded in 84.34898755000904s: True [2018-02-20 08:16:49,806: DEBUG/MainProcess] | Worker: Stopping Hub... [2018-02-20 08:16:49,806: DEBUG/MainProcess] | Consumer: Shutdown Gossip... [2018-02-20 08:16:49,806: DEBUG/MainProcess] | Consumer: Shutdown Heart... [2018-02-20 08:16:49,806: DEBUG/MainProcess] | Consumer: Shutdown Control... [2018-02-20 08:16:49,806: DEBUG/MainProcess] | Consumer: Shutdown Tasks... [2018-02-20 08:16:49,806: DEBUG/MainProcess] Canceling task consumer... [2018-02-20 08:16:49,806: DEBUG/MainProcess] Closing consumer channel... [2018-02-20 08:16:49,807: DEBUG/MainProcess] | Consumer: Shutdown Events... [2018-02-20 08:16:49,808: DEBUG/MainProcess] Closed channel #1 [2018-02-20 08:16:49,808: DEBUG/MainProcess] | Consumer: Shutdown Connection... [2018-02-20 08:16:49,809: DEBUG/MainProcess] Closed channel #1 [2018-02-20 08:16:49,810: DEBUG/MainProcess] removing tasks from inqueue until task handler finished