nornir-automation / nornir

Pluggable multi-threaded framework with inventory management to help operate collections of devices
https://nornir.readthedocs.io/
Apache License 2.0
1.4k stars 237 forks source link

grouped tasked stop if any grouped tasks fail #699

Closed dmzook closed 3 years ago

dmzook commented 3 years ago

There seems to be no way to continue grouped tasks if any of them fail. On the main task you can use 'on_failed=True' but that does not seem to work for grouped tasks.

def sub_task1(task: Task) -> Result:
    return Result(host=task.host, failed=True)

def sub_task2(task: Task) -> Result:
    return Result(host=task.host, result='Hello')

def main_task(task: Task) -> Result:
    task.run(task=sub_task1) #fails on purpose
    task.run(task=sub_task2, on_failed=True) # does not get called

nr.run(task=main_task)

Here I believe is the problem.

        r = run_task.start(self.host)
        self.results.append(r[0] if len(r) == 1 else cast("Result", r))

        if r.failed:
            # Without this we will keep running the grouped task
            raise NornirSubTaskError(task=run_task, result=r)

        return r

It will always kill a failed subtask and not allow more subtasks to be run.

dbarrosop commented 3 years ago

Subtasks are treated like regular functions so if you don't want them to trigger a failure you should use try...except....

def main_task(task: Task) -> Result:
    try:
        task.run(task=sub_task1) #fails on purpose
    except Exception as e:
        # I actually don't care if this happens and want to continue
        pass
    task.run(task=sub_task2)

As a matter of fact, a subtask failing shouldn't flag the host as failed. Only "main" tasks failures will, that is done by design so you can implement your own recovery logic/mechanisms easily.