materialsproject / jobflow

jobflow is a library for writing computational workflows.
https://materialsproject.github.io/jobflow
Other
90 stars 25 forks source link

Make `run_locally(ensure_success=True)` raise the actual encountered exception #611

Open janosh opened 1 month ago

janosh commented 1 month ago

when debugging a failing workflow with run_locally, it can be very helpful to have access to the exception object. currently, you always get the same generic error:

https://github.com/materialsproject/jobflow/blob/e61b50fc864dd20221adb3c331b5887a3c184e42/src/jobflow/managers/local.py#L181

to see what went wrong, you have to run with

run_locally(log=True, ensure_success=True)

and then hunt for the error message in the logs which can be verbose

how about we raise the actual error message instead of this generic RuntimeError? a (possibly bad style) sketch of how to implement this:

Screenshot 2024-05-14 at 16 48 48

for the above to work, _run_job would have to be modified to return the caught exception:

Screenshot 2024-05-14 at 16 54 45

janosh commented 1 month ago

pinging @utf @Andrew-S-Rosen @JaGeo in case you have thoughts?

jmmshn commented 4 weeks ago

Second this.

Also, The raise_immediately flag kind of handles this: https://github.com/materialsproject/jobflow/blob/e61b50fc864dd20221adb3c331b5887a3c184e42/src/jobflow/managers/local.py#L24

I feel real-world exceptions are so deep down the call stack that reading things becomes problematic. I have been running this flag and

ipython --pdb test.py

to debug workflows and it's been pretty effective.

janosh commented 4 weeks ago

thanks for the tip! i'll give the raise_immediately=True + ipython --pdb combo a try