stan-dev / cmdstanr

CmdStanR: the R interface to CmdStan
https://mc-stan.org/cmdstanr/
Other
143 stars 63 forks source link

Store the information of how CmdStan was called #900

Open avehtari opened 9 months ago

avehtari commented 9 months ago

Sometimes for debugging purposes or in order to repeat the same call from commandline, it would be useful if CmdStanFit object would include the actual options used to call CmdStan

jgabry commented 9 months ago

By 'options used to call CmdStan' do you mean something different than what is returned as part of fit$metadata()?

Edit: here's an example of what is currently there:

fit <- cmdstanr_example()
str(fit$metadata())
List of 40
 $ stan_version_major  : num 2
 $ stan_version_minor  : num 33
 $ stan_version_patch  : num 0
 $ start_datetime      : chr "2024-01-23 18:34:47 UTC"
 $ method              : chr "sample"
 $ save_warmup         : num 0
 $ thin                : num 1
 $ gamma               : num 0.05
 $ kappa               : num 0.75
 $ t0                  : num 10
 $ init_buffer         : num 75
 $ term_buffer         : num 50
 $ window              : num 25
 $ algorithm           : chr "hmc"
 $ engine              : chr "nuts"
 $ metric              : chr "diag_e"
 $ stepsize_jitter     : num 0
 $ num_chains          : num 1
 $ id                  : num [1:4] 1 2 3 4
 $ init                : num [1:4] 2 2 2 2
 $ seed                : num 8.02e+08
 $ refresh             : num 100
 $ sig_figs            : num -1
 $ profile_file        : chr "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpNmr6Mp/logistic-profile-202401231134-1-0be96e.csv"
 $ stanc_version       : chr "stanc3 v2.33.1"
 $ sampler_diagnostics : chr [1:6] "accept_stat__" "stepsize__" "treedepth__" "n_leapfrog__" ...
 $ variables           : chr [1:105] "lp__" "alpha" "beta[1]" "beta[2]" ...
 $ step_size_adaptation: num [1:4] 0.808 0.689 0.743 0.719
 $ model_name          : chr "logistic_model"
 $ adapt_engaged       : num 1
 $ adapt_delta         : num 0.8
 $ max_treedepth       : num 10
 $ step_size           : num [1:4] 1 1 1 1
 $ iter_warmup         : num 1000
 $ iter_sampling       : num 1000
 $ threads_per_chain   : num 1
 $ time                :'data.frame':   4 obs. of  4 variables:
  ..$ chain_id: num [1:4] 1 2 3 4
  ..$ warmup  : num [1:4] 0.023 0.023 0.024 0.021
  ..$ sampling: num [1:4] 0.07 0.075 0.074 0.072
  ..$ total   : num [1:4] 0.093 0.098 0.098 0.093
 $ stan_variable_sizes :List of 4
  ..$ lp__   : num 1
  ..$ alpha  : num 1
  ..$ beta   : num 3
  ..$ log_lik: num 100
 $ stan_variables      : chr [1:4] "lp__" "alpha" "beta" "log_lik"
 $ model_params        : chr [1:105] "lp__" "alpha" "beta[1]" "beta[2]" ...
avehtari commented 9 months ago

Given that metadata, how do I call cmdstan with those options in command line?

jgabry commented 9 months ago

Oh I see, yeah there's no function that would take that info and compose the call to cmdstan.

What about setting options("cmdstanr_verbose"=TRUE), which results in printing the call to cmdstanr when you're running a model. See https://mc-stan.org/cmdstanr/articles/cmdstanr-internals.html#troubleshooting-and-debugging. If you scroll down a bit in that section past the compilation output, the call to cmdstan is printed and you could copy and paste it. Is that sufficient?

Running ./bernoulli 'id=1' random 'seed=90958316' data \
  'file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/standata-742874baeb02.json' \
  output \
  'file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-1-4db120.csv' \
  'profile_file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-profile-202312131009-1-1c1f83.csv' \
  'method=sample' 'num_samples=100' 'num_warmup=100' 'save_warmup=0' \
  'algorithm=hmc' 'engine=nuts' adapt 'engaged=1'
avehtari commented 9 months ago

It would be nice to have cmdstanr_verbose=TRUE as documented argument for $sampler(), $pathfinder(), etc

It could be useful to include that call to cmdstan to the returned object, too