jupyterhub / binderhub

Run your code in the cloud, with technology so advanced, it feels like magic!
https://binderhub.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.54k stars 388 forks source link

Check for low-level errors when submitting builds #1517

Closed manics closed 2 years ago

manics commented 2 years ago

The builder uses pool.submit to submit a task, but it doesn't check that the task was successfully submitted. Instead it relies on progress events being pushed to a shared queue. If there is an error in the task it will hang forever waiting for a non-existing item to be pushed to the queue.

add_done_callback will be called when the task is finished.

Since these are likely to be internal BinderHub development errors this just generates an exception in the logs. In theory it should be possible to push an event that the user can see, but when I tried this it didn't work- I think there may be a race condition between the task ending and the events appearing in the queue, and given that it's been fine up to now I think just exposing it in the BinderHub logs is good enough.

For example, modify FakeBuild.stream_logs to include print(non_existent_variable). Without this PR there is no indication that anything is wrong.