search

databricks / run-notebook

Apache License 2.0
49 stars 19 forks source link

Access output even when job fails #45

Open kasuteru opened 1 year ago

kasuteru commented 1 year ago

Currently, when the job fails, the outputs are not written - but this is exactly when they are interesting! (Especially the link to the run url). Ideally, a 5th output about the success of the notebook (read from the jobs run API - SUCCESS / FAILURE / CANCELLED...) would also report state.

Use case for this is to notify, in follow-up step, the team that something went wrong, directly providing them with the link to the notebook and the error message.

Is this repository still maintained? Is there a chance to get this added? Or at least, to get it merged? I am asking because the last change was made in 2022...

Edit: What I meant by 5th output would be result_state:

image

kasuteru commented 1 year ago

Edit: I managed to implement this for run_url, which for my use case is the relevant variable that I need in case a job fails. I integrated this in my workflow and it actually works.

If there is any interest from the authors of this repository, I can provide the code as a PR (it's just a few lines of change). Definitely it is possible to have the other outputs as well.

~After trying to implement this, it unfortunately seems like github actions doesn't allow setting outputs when errors arise, so it looks like this needs to be a custom solution. If I find something worth merging I will write a PR but for now I will copy this action and adapt it to our needs.~