Use nonzero exit code if project health is not 100%

[DanBurton]

I'm running stan in a CI context and I'd like the command to "fail" if project health is not 100% so that CI will notify me that such is the case.

Expected: non-zero exit code.

Actual:

$ stan

┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━┫
┃ Project health               ┃ 97.14% ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━┛
$ echo $?
0

[chshersh]

Current Stan behaviour is to exit with non-zero code when at least one observation has the Error severity. But we do have plans to implement support for defining custom failure rules, so users can specify when they want non-zero exit code: on specific health threshold, on particular severity, on triggering specific inspections, etc. Failing on non-100% health is not always desirable and doesn't fit everyone, so that's why we are thinking about allowing more custom settings.

The following issue covers this use case:

• https://github.com/kowainik/stan/issues/246

[DanBurton]

Thank you for the explanation! I was easily able to fork Stan to get the behavior I want, but I look forward to this configuration option in the future so that the fork is unnecessary.