search

DurhamARC / ManyFEWS

The Many Flood Early Warning System
GNU General Public License v3.0
7 stars 3 forks source link

Feed back task success/failure status to admin user interface #83

Closed sjmf closed 1 year ago

sjmf commented 2 years ago

There is currently no display of whether a task has succeeded or failed in the UI. Per #82, errors can be generated which are only visible in the Celery logs. Whether a task has succeeded or failed (and if so, with what error?) is a key piece of information which needs to be visible to the end user.

Task status can be retrieved from Celery. Implementation details will need to be determined but are likely to involve changes to the UI and supporting infrastructure.

sjmf commented 2 years ago

Further, the "Task last run" display in the Admin UI is currently blank. Is this populated on a successful completion of the task? And could it be used to indicate failure status too?

sjmf commented 1 year ago

Today's investigation turns over the following:

  1. Celery requires a separate scheduler to run to enable 'beat' of scheduled tasks, as described in the celery documentation. The commit 30d5be2 addresses this by adding the configuration to docker-compose.yml
  2. With the scheduler running, I observed the 'run once' task to set up the model was automatically scheduled and ran successfully. However, the other periodic tasks were not scheduled or run automatically. This could be related to issue 413 on the django-celery-beat repo. This issue has been open for a full year, which is worrying as it is unclear that the project has the developer resource to actually address these issues. There are comments on the linked issue relating to the same.
  3. The module django-celery-results can be used to display task status. However, the results are displayed as binary hashes. I was able to display these in JSON by changing the serialization type, but this is not a permanent fix: Alison left a comment in celery.py#L19 that numpy arrays do not correctly serialise with JSON.
sjmf commented 1 year ago

Debugging (2) in the above list: I set a periodic task of 1 minute, which runs successfully. However, it takes a little while for Last Run Datetime to be populated. The field is not set if the task is launched manually, but is set if the task runs by itself.

I have added commit caf8aa5 which adds django-celery-results. While this is not a perfect solution, it will enable end-users to at least see what tasks have run and when.