MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

add screen to view current background tasks #154

Closed ghukill closed 6 years ago

ghukill commented 6 years ago

Background tasks are run with Django-Background-Tasks, as configured in the file tasks.py.

Tasks are stored in the DB, and it would be fairly trivial to create a view in Combine to see currently running jobs (any completed jobs perhaps). Datatables might be a natural fit, as these would likely be small enough in number the data could be transferred wholesale via the Django view -- no need for an AJAX endpoint -- and munged with DT.

This might be more handy with the envisioned expansion of background tasks to include bulk data downloading and indexing.

Finally, would likely need some translation for tasks. An example row from SQL looks like this:

mysql> select * from background_task;
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+-----------+-------------------+-------------------------+
| id | task_name             | task_params | task_hash                                | verbose_name | priority | run_at                     | repeat | repeat_until | queue | attempts | failed_at | last_error | locked_by | locked_at | creator_object_id | creator_content_type_id |
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+-----------+-------------------+-------------------------+
| 86 | core.tasks.job_delete | [[127], {}] | bb860e7905e55fee57d02540d74902df0bb6d373 | NULL         |        0 | 2018-04-16 12:16:03.857000 |      0 | NULL         | NULL  |        0 | NULL      |            | NULL      | NULL      |              NULL |                    NULL |
| 87 | core.tasks.job_delete | [[128], {}] | bf613ed6b6c147068d9413d53140db8f33bf30cf | NULL         |        0 | 2018-04-16 12:16:04.009364 |      0 | NULL         | NULL  |        0 | NULL      |            | NULL      | NULL      |              NULL |                    NULL |
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+-----------+-------------------+-------------------------+

But with the task_name in hand, could even ping the tasks.py file for some kind of human translation. Perhaps a default argument for the task, much like is the case for validation tests.

ghukill commented 6 years ago

And here are completed tasks examples:

mysql> select * from  background_task_completedtask limit 2;
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+----------------------------+-------------------+-------------------------+
| id | task_name             | task_params | task_hash                                | verbose_name | priority | run_at                     | repeat | repeat_until | queue | attempts | failed_at | last_error | locked_by | locked_at                  | creator_object_id | creator_content_type_id |
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+----------------------------+-------------------+-------------------------+
|  1 | core.tasks.job_delete | [[4], {}]   | 002511a1ff001d87cf3ca43bbdd4cdae3d6b1125 | NULL         |        0 | 2018-03-21 17:47:31.469987 |      0 | NULL         | NULL  |        1 | NULL      |            | 31275     | 2018-03-21 17:47:31.350985 |              NULL |                    NULL |
|  2 | core.tasks.job_delete | [[3], {}]   | fd64e8646e158d150813f547ca74098b47e5c144 | NULL         |        0 | 2018-03-21 17:47:32.458507 |      0 | NULL         | NULL  |        1 | NULL      |            | 31275     | 2018-03-21 17:47:32.340486 |              NULL |                    NULL |
+----+-----------------------+-------------+------------------------------------------+--------------+----------+----------------------------+--------+--------------+-------+----------+-----------+------------+-----------+----------------------------+-------------------+-------------------------+
ghukill commented 6 years ago

Done, added entirely new CombineBackgroundTasks model to manage them.