search

thieman / dagobah

Simple DAG-based job scheduler in Python
Do What The F*ck You Want To Public License
755 stars 160 forks source link

Add automated task retrying #20

Open thieman opened 11 years ago

thieman commented 11 years ago

Some tasks have a tendency to break (I am thinking specifically of data processes reliant on shaky third party APIs) but it may be difficult to properly account for all of the potential errors ahead of time and make sure your task can survive it. In this situation, it can be helpful for Dagobah to automatically retry your job up to a certain number of times before giving up and declaring the task a failure.

I think we need to do three things:

  1. Add a user-configurable maximum number of times to retry a Task, set at the Task level
  2. Add a property on the Task object that tracks how many times the Task was retried during its last run. This will end up in the run log.
  3. Patch the existing email templates to show task retries.
Heinoman commented 6 years ago

Is there any progress on this issue yet?

thieman commented 6 years ago

I am not aware of any work. I personally am no longer contributing to this project, any work is probably being done on forks.