Added cleanup on pipeline failure to project uploads

comses / miracle

Repeatable data analysis workflows for computational models

1 stars 3 forks source link

Added cleanup on pipeline failure to project uploads #40

Closed cpritcha closed 8 years ago

cpritcha commented 8 years ago

If the project pipeline fails for any reason the project's filesystem remnants in the packrat, project and archive directories are deleted.

alee commented 8 years ago

Hey Calvin, it looks like you moved the pipeline execution from celery back into Django, was this intended?

cpritcha commented 8 years ago

I combined everything into one Celery task (run_metadata_pipeline) because dealing with resource handling on a Celery chain would have been more complex (I could not come up with a way other than breaking up the chain into pieces and adding more than one error handler which I though was ugly).

cpritcha commented 8 years ago

So the pipeline execution should still be happening in Celery