search

StanJulia / StanSample.jl

WIP: Wrapper package for the sample method in Stan's cmdstan executable.
MIT License
18 stars 4 forks source link

How can I view the HMC message while fitting the model? #46

Closed zhengchencai closed 2 years ago

zhengchencai commented 2 years ago

Hi there, I am new to StanSample I wonder how can I show the messages while fitting the model like below as if we do in cmdstanr? These messages are quite important. Thanks a lot.

Running MCMC with 4 parallel chains...

Chain 1 Iteration: 1 / 2000 [ 0%] (Warmup) Chain 1 Iteration: 500 / 2000 [ 25%] (Warmup) Chain 1 Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 1 Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 1 Iteration: 1500 / 2000 [ 75%] (Sampling) Chain 1 Iteration: 2000 / 2000 [100%] (Sampling) Chain 2 Iteration: 1 / 2000 [ 0%] (Warmup) Chain 2 Iteration: 500 / 2000 [ 25%] (Warmup) Chain 2 Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 2 Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 2 Iteration: 1500 / 2000 [ 75%] (Sampling) Chain 2 Iteration: 2000 / 2000 [100%] (Sampling) Chain 3 Iteration: 1 / 2000 [ 0%] (Warmup) Chain 3 Iteration: 500 / 2000 [ 25%] (Warmup) Chain 3 Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 3 Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 3 Iteration: 1500 / 2000 [ 75%] (Sampling) Chain 3 Iteration: 2000 / 2000 [100%] (Sampling) Chain 4 Iteration: 1 / 2000 [ 0%] (Warmup) Chain 4 Iteration: 500 / 2000 [ 25%] (Warmup) Chain 4 Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 4 Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 4 Iteration: 1500 / 2000 [ 75%] (Sampling) Chain 4 Iteration: 2000 / 2000 [100%] (Sampling) Chain 1 finished in 0.0 seconds. Chain 2 finished in 0.0 seconds. Chain 3 finished in 0.0 seconds. Chain 4 finished in 0.0 seconds.

All 4 chains finished successfully. Mean chain execution time: 0.0 seconds. Total execution time: 0.4 seconds.

goedman commented 2 years ago

These are typically in the .log file in the tmp directory (in the same directory where the .stan file is written).

Why do you think they are important (just for my education)? If you use multiple threads, they are intermingled.

Rob

Edit: In the REPL you can use e.g.: ;cat "$(joinpath(sm.tmpdir, sm.name*"_log_1.log"))"

The ";" puts the REPL in 'shell' mode. 'sm' is the stanmodel.

zhengchencai commented 2 years ago

Thanks for your reply @goedman . They are important because 1) I know immediately the fitting time, 2) if multiple chains are behaving differently such as 2 chains finished but the other 2 are still sampling let's say at 60%, then it means either the model might be sensitive to initialization or there are pathological behavior of the model, 3) gradient evaluation took A seconds, and 1: 1000 transitions using 10 leapfrog steps per transition would take B seconds, these are printed at the beginning giving users a basic idea of how computationally expensive is the model, some times if I reparameterize the model these numbers drops a lot, 4) warnings showing problematic rhat, ESS and divergent transitions right after fitting. I guess these rich messages are considered as advantages of cmdstan even when compared with rstan.

goedman commented 2 years ago

That makes sense. Thanks.