insitro / redun

Yet another redundant workflow engine
https://insitro.github.io/redun/
Apache License 2.0
510 stars 43 forks source link

is there a way for an error to not stop redun from running non-dependent tasks? #97

Closed njbernstein closed 3 months ago

njbernstein commented 3 months ago

e.g. pipeline is task1->task2->task3 task1->task4->task5

If task2 fails and returns an error I still want redun to run task5 if task4 was successful

mattrasmus commented 3 months ago

You could look into using catch() for catching an error and returning a intermediate value that allows the workflow to continue. See https://github.com/insitro/redun/blob/367033e87ad17ad8a1bdb09bec472f846ffe4b4e/examples/07_control_flow/workflow.py#L36-L42

So your example would look something like:

@task
def task1():
    x = catch(task3(task2()), ValueError, None)
    y = task5(task4())
    return [x, y]
njbernstein commented 3 months ago

@mattrasmus Great! Thanks!