scimma / blast

Django web app for the automatic characterization of supernova hosts
MIT License
1 stars 3 forks source link

Automatically restart incomplete workflows #245

Open manning-ncsa opened 4 months ago

manning-ncsa commented 4 months ago

Sometimes workflows fail to complete due to failures of the workflow management system, for example when a Celery worker dies or there is a problem with RabbitMQ. We should create a periodic task to scan for incomplete workflows and retrigger them automatically until they are complete (i.e. none of the workflow tasks have status "not processed").