Error in `read_csv_as_stanfit` for variational approximations with latest Cmdstan version

[nicholasjclark]

I believe I may have found a bug in the conversion of cmdstan_csv files to stanfit objects for the latest version of Cmdstan (version 2.35.0) and the latest version of brms (version 2.21.0). The reprex below illustrates how there is an ERROR somewhere around this line in read_csv_as_stanfit:

fit1 <- brm(
      count ~ zBase * Trt + (1|patient),
      data = epilepsy, family = poisson(),
      prior = prior(normal(0, 10), class = b) +
          prior(cauchy(0, 2), class = sd),
      backend = 'cmdstanr',
      algorithm = 'meanfield'
 )

Start sampling
------------------------------------------------------------ 
EXPERIMENTAL ALGORITHM: 
  This procedure has not been thoroughly tested and may be unstable 
  or buggy. The interface is subject to change. 
------------------------------------------------------------ 
Gradient evaluation took 5.9e-05 seconds 
1000 transitions using 10 leapfrog steps per transition would take 0.59 seconds. 
Adjust your expectations accordingly! 
Begin eta adaptation. 
Iteration:   1 / 250 [  0%]  (Adaptation) 
Iteration:  50 / 250 [ 20%]  (Adaptation) 
Iteration: 100 / 250 [ 40%]  (Adaptation) 
Iteration: 150 / 250 [ 60%]  (Adaptation) 
Iteration: 200 / 250 [ 80%]  (Adaptation) 
Success! Found best value [eta = 1] earlier than expected. 
Begin stochastic gradient ascent. 
  iter             ELBO   delta_ELBO_mean   delta_ELBO_med   notes  
   100       -15951.188             1.000            1.000 
   200       -14930.295             0.534            1.000 
   300       -14491.509             0.049            0.068 
   400       -14176.069             0.026            0.030 
   500       -13941.417             0.020            0.022 
   600       -14716.655             0.035            0.053 
   700       -13284.329             0.080            0.108 
   800       -13657.967             0.068            0.108 
   900       -13281.227             0.028            0.028 
  1000       -15733.418             0.092            0.156 
  1100        -4994.938             1.153            2.150   MAY BE DIVERGING... INSPECT ELBO 
  1200        -3046.309             1.395            2.150   MAY BE DIVERGING... INSPECT ELBO 
  1300        -1868.384             0.635            0.640   MAY BE DIVERGING... INSPECT ELBO 
  1400        -1310.923             0.528            0.630   MAY BE DIVERGING... INSPECT ELBO 
  1500        -1019.463             0.356            0.425 
  1600         -855.615             0.239            0.286 
  1700         -790.732             0.137            0.191 
  1800         -758.822             0.062            0.082 
  1900         -739.077             0.034            0.042 
  2000         -740.193             0.014            0.027 
Informational Message: The maximum number of iterations is reached! The algorithm may not have converged. 
This variational approximation is not guaranteed to be meaningful. 
Drawing a sample of size 1000 from the approximate posterior...  
COMPLETED. 
Finished in  0.4 seconds.
Error in `[.data.frame`(diagnostics[[i]], rstan_diagn_order) : 
  undefined columns selected

Here are my package / software versions:

packageVersion('brms')
[1] ‘2.21.0’

packageVersion('cmdstanr')
[1] ‘0.8.1’

cmdstanr::cmdstan_version()
[1] "2.35.0"

version
               _                                
platform       x86_64-w64-mingw32               
arch           x86_64                           
os             mingw32                          
crt            ucrt                             
system         x86_64, mingw32                  
status                                          
major          4                                
minor          4.1                              
year           2024                             
month          06                               
day            14                               
svn rev        86737                            
language       R                                
version.string R version 4.4.1 (2024-06-14 ucrt)
nickname       Race for Your Life   

[paul-buerkner]

I think this issue has been already fixed in brms. Can you test it with the latest github version of brms?

[nicholasjclark]

Yes that sorted it. Thanks very much @paul-buerkner