[Python] cmdstanpy warnings

[tcuongd]

This is an FYI / guidance for those who are currently running cmdstanpy>=1.0.2, specifically when doing full MCMC sampling (mcmc_samples > 0). We've noticed that cmdstanpy now logs warnings fairly often, even when overall the sampling is healthy and there are no divergences. Details are provided here: https://github.com/stan-dev/cmdstanpy/issues/584

The warnings you may see are:

1) Hitting max treedepth:

23:23:47 - cmdstanpy - WARNING - Some chains may have failed to converge.
    Chain 1 had 5 iterations at max treedepth (3.3%)
    Chain 2 had 1 iterations at max treedepth (0.7%)
    Chain 3 had 13 iterations at max treedepth (8.7%)
    Chain 4 had 2 iterations at max treedepth (1.3%)

2) Warnings about variable values

23:23:47 - cmdstanpy - WARNING - Non-fatal error during sampling:
Exception: normal_id_glm_lpdf: Scale vector is 0, but must be positive finite! (in '/Users/runner/work/prophet/prophet/python/stan/prophet.stan', line 137, column 2 to line 142, column 4)
Exception: normal_id_glm_lpdf: Matrix of independent variables is inf, but must be finite! (in '/Users/runner/work/prophet/prophet/python/stan/prophet.stan', line 137, column 2 to line 142, column 4)

What to do

These warnings may be false positives and the MCMC sampling may have converged just fine. To confirm this, run:

print(m.stan_fit.diagnose())

If you see output similar to this:

Processing csv files: /var/folders/3x/ypx818511wsdnxc3knn7yy1r0000gn/T/tmpj9urc8se/prophet_modela2q2ppg4/prophet_model-20220629231921_1.csv, /var/folders/3x/ypx818511wsdnxc3knn7yy1r0000gn/T/tmpj9urc8se/prophet_modela2q2ppg4/prophet_model-20220629231921_2.csv, /var/folders/3x/ypx818511wsdnxc3knn7yy1r0000gn/T/tmpj9urc8se/prophet_modela2q2ppg4/prophet_model-20220629231921_3.csv, /var/folders/3x/ypx818511wsdnxc3knn7yy1r0000gn/T/tmpj9urc8se/prophet_modela2q2ppg4/prophet_model-20220629231921_4.csv

Checking sampler transitions treedepth.
21 of 600 (3.5%) transitions hit the maximum treedepth limit of 10, or 2^10 leapfrog steps.
Trajectories that are prematurely terminated due to this limit will result in slow exploration.
For optimal performance, increase this limit.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory for all transitions.

Effective sample size satisfactory.

Split R-hat values satisfactory all parameters.

Processing complete.

the sampling procedure likely succeeded and you can trust the results.

If you would like to only show logs from cmdstanpy when there is an error (instead of just a warning), you can set the logging level for cmdstanpy in your script:

import logging
logger = logging.getLogger('cmdstanpy')
logger.setLevel(logging.ERROR)

[tcuongd]

@andrjohns not sure if you ever saw this warning:

23:23:47 - cmdstanpy - WARNING - Non-fatal error during sampling:
Exception: normal_id_glm_lpdf: Scale vector is 0, but must be positive finite! (in '/Users/runner/work/prophet/prophet/python/stan/prophet.stan', line 137, column 2 to line 142, column 4)
Exception: normal_id_glm_lpdf: Matrix of independent variables is inf, but must be finite! (in '/Users/runner/work/prophet/prophet/python/stan/prophet.stan', line 137, column 2 to line 142, column 4)

when testing the normal_id_glm specification for the stan model. It doesn't seem to actually affect sampling, but maybe there's a quick way to prevent it from showing.

[skannan-maf]

Thank you brother! Very useful!

[AshtonCoop]

Hi there, I got value error after running print(m.stan_fit.diagnose())

No CmdStan installation found, run command "install_cmdstan"or (re)activate your conda environment!

However, I checked my conda environment, cmdstan or cmdstanpy is installed. What could be wrong here? Thanks!