botify-labs / simpleflow

Python library for dataflow programming.
https://botify-labs.github.com/simpleflow/
MIT License
68 stars 24 forks source link

Improve error handling and helpers for failed tasks #416

Open ybastide opened 1 year ago

ybastide commented 1 year ago

Context

We noticed it would be helpful, when a workflow fails, to have more information on the underlying activity and error details to provide helpful information to help diagnosing why the workflow failed.

Tech details

Current approach

When handling a failure in a workflow, we have access to the failure context via simpleflow.task.TaskFailureContext, which exposes:

Possible improvements

It would be helpful to have access to additional information regarding the underlying exception in a straightforward way, with all fields at the same level, e.g.:

The resulting improvements should be documented under the Features / Error Handling section.