search

jupyterhub / batchspawner

Custom Spawner for Jupyterhub to start servers in batch scheduled systems
BSD 3-Clause "New" or "Revised" License
190 stars 134 forks source link

Fail hard on first error in batch script #177

Closed yuvipanda closed 4 years ago

yuvipanda commented 4 years ago

Bash generally keeps executing code line by line even if any of the lines fail. For example, if your path is not set properly, batchspawner-singleuser will not be found by which. This should terminate execution. However, without set -e, execution will just continue and fail with a different error.

This patch does set -euo pipefail for all submission scripts. This causes them to fail on:

  1. Any non-zero command return code (-e)
  2. On undefined environment variables (-u)
  3. Treat failures in any part of a pipeline as failure, rather than just the last command (-o pipefail)
rkdarst commented 4 years ago

This is a good idea, and probably should have been done before. Could you add a note to the changelog as a potentially breaking change? At least in my prologue, I run stuff for debugging that is expected to fail (echoing environment variables), and I wouldn't be surprised if others do, too. (I can also add the note myself).

It could be set in the default prologue, so that the minimum amount is hardcoded (of course, if someone wants it off then they can disable in their own prologue. Maybe it's better to have someone explicitely disable, than risk turning it off. I'd recommend to leave it as is.)

Thanks!

mbmilligan commented 4 years ago

Agree, this seems like a sensible change. Likewise agree with the reasoning about putting it in the base templates rather than the prologue.

welcome[bot] commented 4 years ago

Congrats on your first merged pull request in this project! :tada: congrats Thank you for contributing, we are very proud of you! :heart:

rkdarst commented 4 years ago

I just realized an issue here, some of the scripts use /bin/sh, and it seems -o pipefail is a bashism. At least on my Ubuntu it is.