search

brian-watkins / elm-spec

Describe the behavior of Elm programs
https://package.elm-lang.org/packages/brian-watkins/elm-spec/latest/
MIT License
29 stars 4 forks source link

elm-spec-runner fails with exit code 0 #29

Closed mkaszas closed 3 years ago

mkaszas commented 3 years ago

When elm-spec-runner fails to compile the project, or specs are rejected, the exit code of the process is 0. This makes it hard to use the command in a CI pipeline.

Current behaviour

➜ elm-spec
Compiling Elm ... Done!

Running specs: x.......................................... (1.8s)

Accepted: 42
Rejected: 1

Failed to satisfy spec:
...

➜ echo $?
0

Expected behaviour

When these errors occur, exit with code 1.

Possible solution

In consoleReporter.js, the finish() method can be modified to call process.exit(1) on non-happy paths.

brian-watkins commented 3 years ago

Great suggestion! I'll take a look and hopefully can get something out soon.

brian-watkins commented 3 years ago

I released a new version of elm-spec-runner (v2.2.0) that includes a fix for this. Now, if any observations are rejected (and elm-spec-runner is not being run in watch mode) then the exit code will be 1. If an error occurs that prevents the spec suite from executing, the exit code will be 2.

Thanks again for the feature suggestion, and please feel free to share any more feedback you might have.

mkaszas commented 3 years ago

Thank you! Just tested the new version and it works great. 🎉

Thanks for the quick reply, and for maintaining this package!