search

RealOrangeOne / django-tasks

A reference implementation and backport of background workers and tasks in Django
https://pypi.org/project/django-tasks/
BSD 3-Clause "New" or "Revised" License
211 stars 13 forks source link

Queue retention #16

Open RealOrangeOne opened 1 month ago

RealOrangeOne commented 1 month ago

The task queue doesn't have the ability to remove old entries. This means the table can grow considerably, impacting performance.

"NEW" and "PROCESSING" tasks should never be removed! I suspect retention for complete and failed tasks should be configurable separately (even if failed inherits completed by default).

jribbens commented 1 month ago

Removing expired entries from a table is a classic case of something that requires a background worker. Possibly you could look into seeing if there is some sort of background worker system available for Django to carry out this task?