search

stan-dev / cmdstan

CmdStan, the command line interface to Stan
https://mc-stan.org/users/interfaces/cmdstan
BSD 3-Clause "New" or "Revised" License
207 stars 90 forks source link

diagnose exit code #1278

Open jsks opened 1 month ago

jsks commented 1 month ago

Currently, the diagnose utility prints all output to stdout and returns 0 regardless of warnings. Would it make sense to return a non-zero exit status if there's an error with any of the checks?

This would make it easier to programmatically distinguish CmdStan runs with diagnostic warnings in the shell. As far as I know, currently the diagnose output needs to be checked manually or the output loaded into one of the programming language specific packages.

The return line in diagnose.cpp can just return the bool value from the variable has_error, or a more specific exit code can be used.

mitzimorris commented 1 month ago

the diagnose command detects potential errors - it returns warnings. however, your point is valid - the return code could be used to signal warnings / no_warnings.

regarding the diagnose utility - I think that it's probably due for updates - it was written circa 2016 and reflects the thinking then. cf https://discourse.mc-stan.org/t/is-the-diagnose-utility-deprecated/2312. there are a bunch of case studies by Michael Betancourt on these diagnostics. also, some discussion in the "Visualization in Bayesian Workflow" paper of Gabry et al - https://arxiv.org/abs/1709.01449

WardBrian commented 1 month ago

I think it would make sense to something similar to grep:

0 - no problem 1 - warning raised 2 - internal error/exception, not a problem with this specific set of draws