Email tasks are not written to the results database, nor can they be tracked

[simondrabble]

Issue:

Sending email via django-celery-email, results cannot be tracked, leading to issues with e.g. the following scenario:

1. A user submits (via a django form) a support request.
2. The request is emailed (from the backend) to interested parties.
3. The email result cannot be tracked, thus it is impossible to provide feedback to the user.

Non-django celery email tasks do not exhibit this behaviour.

Config:

# pip freeze|grep -i 'django\|celery'
celery==4.2.0
Django==1.11.4
django-celery-email==2.0.0

CELERY_EMAIL_TASK_CONFIG = {
       'task_serializer': 'json',
       'result_serializer': 'json', 
       'name': 'djcelery_email_send_multiple',
       'ignore_result': False, 
       'result_backend': 'db+postgresql://****'
 }

Procedure:

Send an email (e.g. via mail.send_mail(...))

The email is delivered successfully, i.e. the task completes.

Observed:

Resulting task has an ID but no entry in the database:

ipdb> task
<AsyncResult: 1d7ac887-fc81-42c7-b6c7-a77449d4bf9f>

$  echo 'select * from celery_taskmeta;' | ./manage.py dbshell
 id | task_id | status | result | date_done | traceback
----+---------+--------+--------+-----------+-----------
(0 rows)

Task cannot be waited for, and status is not updated:

pdb> task.state
u'PENDING'
ipdb> task.wait()      # I killed it here after ~30 seconds
^C*** KeyboardInterrupt:
ipdb> task.state
u'PENDING'

Expected:

The task is written to the database, and the status is updated to SUCCESS or FAILURE as appropriate.

Additional Info:

The problem also occurs if the django-celery-email task is run directly:

ipdb> mess = email_to_dict(msg)
ipdb> results = send_emails.delay(mess)
ipdb> results
<AsyncResult: 6fd42b8c-b2e7-4576-af89-f46304e04cff>
ipdb> results.status
u'PENDING'

It does not occur if a non-email task is executed:

ipdb> from myapp.celery import debug_task
ipdb> t = debug_task.delay()
ipdb> t
<AsyncResult: 8baed263-7269-43e4-88de-e9fc6d8828aa>
ipdb> t.status
u'SUCCESS'

$ echo 'select * from celery_taskmeta;' | ./manage.py dbshell
 id |               task_id                | status  | result |         date_done          | traceback
----+--------------------------------------+---------+--------+----------------------------+-----------
 16 | 8baed263-7269-43e4-88de-e9fc6d8828aa | SUCCESS |        | 2018-06-26 16:11:08.613751 |

Update: added software versions

[yybot1]

same here, I use django-celery-results and sending email results are not written to database.

[johnthealy3]

Put the following in settings.py and django-celery-results should work with one TaskResult per email:

CELERY_EMAIL_TASK_CONFIG = {'ignore_result': False} CELERY_EMAIL_CHUNK_SIZE = 1

It took me a long time to figure this out even though it is mentioned in the README. Perhaps an update there to explicitly call out the django-celery-results package would be helpful?