Error using example data

[abfleishman]

I just found your package and I am super excited to try it! I installed cmdstanr using the instructions laid out on their page. installed flocker and tried using the data in the tutorial to create a model. it failed immediately. I am guessing it is a problem with my environment? but would love your advice for debugging.

library(flocker)
ex_data <- example_flocker_data()
names(ex_data)
flocker_data <- make_flocker_data(obs = ex_data$obs, 
                                  unit_covs = ex_data$unit_covs, 
                                  event_covs = ex_data$event_covs)

flock(f_occ = ~ uc1, 
      f_det = ~ 1, 
      flocker_data = flocker_data)

Compiling Stan program... SYNTAX ERROR, MESSAGE(S) FROM PARSER: Variable "real" does not exist. error in 'model48c840712d1c_ffd255f5dc30de135393ae897005a9ea' at line 25, column 8

23: 
24:   // Initialize and compute log-likelihood
25:     real lp = 0;
           ^
26:     for (i in 1:vint1[1]) {

---

Error in stanc(file = file, model_code = model_code, model_name = model_name, : failed to parse Stan model 'ffd255f5dc30de135393ae897005a9ea' due to the above error.

sessionInfo()

R version 4.1.0 (2021-05-18) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] flocker_0.1-0

loaded via a namespace (and not attached): [1] nlme_3.1-152 matrixStats_0.61.0 xts_0.12.1 threejs_0.3.3 rstan_2.21.2 rprojroot_2.0.2 tensorA_0.36.2
[8] tools_4.1.0 backports_1.2.1 utf8_1.2.2 R6_2.5.1 DT_0.19 DBI_1.1.1 mgcv_1.8-35
[15] projpred_2.0.2 colorspace_2.0-2 withr_2.4.2 tidyselect_1.1.1 gridExtra_2.3 prettyunits_1.1.1 processx_3.5.2
[22] Brobdingnag_1.2-6 emmeans_1.6.0 curl_4.3.1 compiler_4.1.0 cli_3.1.0 shinyjs_2.0.0 colourpicker_1.1.1
[29] posterior_1.1.0 scales_1.1.1 dygraphs_1.1.1.6 checkmate_2.0.0 brms_2.16.1 mvtnorm_1.1-3 ggridges_0.5.3
[36] callr_3.7.0 StanHeaders_2.21.0-7 stringr_1.4.0 digest_0.6.28 minqa_1.2.4 base64enc_0.1-3 pkgconfig_2.0.3
[43] htmltools_0.5.2 lme4_1.1-27.1 fastmap_1.1.0 htmlwidgets_1.5.4 rlang_0.4.12 rstudioapi_0.13 shiny_1.7.1
[50] farver_2.1.0 generics_0.1.1 jsonlite_1.7.2 zoo_1.8-9 crosstalk_1.2.0 gtools_3.9.2 dplyr_1.0.7
[57] distributional_0.2.2 inline_0.3.19 magrittr_2.0.1 loo_2.4.1 bayesplot_1.8.1 Matrix_1.3-3 Rcpp_1.0.7
[64] munsell_0.5.0 fansi_0.5.0 abind_1.4-5 lifecycle_1.0.1 stringi_1.7.5 MASS_7.3-54 pkgbuild_1.2.0
[71] plyr_1.8.6 grid_4.1.0 parallel_4.1.0 promises_1.2.0.1 crayon_1.4.2 miniUI_0.1.1.1 lattice_0.20-44
[78] splines_4.1.0 ps_1.6.0 pillar_1.6.4 igraph_1.2.7 boot_1.3-28 estimability_1.3 markdown_1.1
[85] shinystan_2.5.0 codetools_0.2-18 stats4_4.1.0 reshape2_1.4.4 rstantools_2.1.1 glue_1.4.2 V8_3.4.2
[92] RcppParallel_5.1.4 remotes_2.3.0 vctrs_0.3.8 nloptr_1.2.2.2 httpuv_1.6.3 gtable_0.3.0 purrr_0.3.4
[99] assertthat_0.2.1 ggplot2_3.3.5 mime_0.12 xtable_1.8-4 coda_0.19-4 later_1.3.0 rsconnect_0.8.24
[106] tibble_3.1.5 shinythemes_1.2.0 gamm4_0.2-6 ellipsis_0.3.2 bridgesampling_1.1-2

[jsocolar]

In order to use the cmdstanr backend, flocker::flock needs to see the argument backend = "cmdstanr". By default (and this just follows conventions from brms) it will try to use package rstan. So it seems you have a problem with the rstan backend. Try adding backend = "cmdstanr" to you call to flocker::flock, and see if that works. If it doesn't work, let's check if you are able to compile and run any brms models. What happens if you do

library(brms)
fit1 <- brm(time ~ age, data = kidney, family = lognormal(), backend = "cmdstanr")

[abfleishman]

@jsocolar Thank you! That was the issue! (clearly a beginner's problem). I must have missed that in your tutorial.
The following worked for me perfectly!

library(flocker)
ex_data <- example_flocker_data()
names(ex_data)
flocker_data <- make_flocker_data(obs = ex_data$obs, 
                                  unit_covs = ex_data$unit_covs, 
                                  event_covs = ex_data$event_covs)

flock(f_occ = ~ uc1, 
      f_det = ~ 1, 
      flocker_data = flocker_data,
      backend = "cmdstanr")

[jsocolar]

No worries. I'm not sure it's in the tutorial, and I'll make sure to update it! I'll leave this issue open until I do that. brms has so many knobs to twiddle that any feedback about which ones need to be documented for users of flocker who don't already use brms is super helpful.

[joshua-goldberg]

Hi there,

I'm also getting a cryptic error using the example data (as well as a stripped-down dataset of my own). Running:

library(flocker)
ex_data <- example_flocker_data()
flocker_data <- make_flocker_data(obs = ex_data$obs, 
                                  unit_covs = ex_data$unit_covs, 
                                  event_covs = ex_data$event_covs)

flock(f_occ = ~ uc1, 
      f_det = ~ 1, 
      flocker_data = flocker_data)

yields the cryptic: Error in stanc(file = file, model_code = model_code, model_name = model_name, : parser failed badly.

For the moment, I am limited to using rstan as a backend, since CmdStan seems to not play nicely with macOS Ventura and Intel silicon. I'm hoping that issue will be resolved soon (which may go some way to resolving this issue?!), but in the meantime, I'd appreciate getting to mess around with flocker using the rstan backend. Any suggestions on how I might do that?

I have verified that both rstan and brms seem to work as expected. I successfully generated samples for the centered parameterization of the eight schools model in rstan and the kidney example in brms. So to my uneducated eye, I should have all of the ingredients to sample with flocker, but clearly something is amiss. I get the same result when I specify backend = 'rstan' explicitly.

Thanks for your help!

Here's my R session environment for reference:

R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Ventura 13.0

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] brms_2.18.0     Rcpp_1.0.9      flocker_0.1-0   forcats_0.5.2   stringr_1.4.1  
 [6] dplyr_1.0.10    purrr_0.3.5     readr_2.1.3     tidyr_1.2.1     tibble_3.1.8   
[11] ggplot2_3.3.6   tidyverse_1.3.2

loaded via a namespace (and not attached):
  [1] googledrive_2.0.0    colorspace_2.0-3     ellipsis_0.3.2       ggridges_0.5.4      
  [5] estimability_1.4.1   markdown_1.3         base64enc_0.1-3      fs_1.5.2            
  [9] rstudioapi_0.14      farver_2.1.1         rstan_2.26.13        DT_0.26             
 [13] fansi_1.0.3          mvtnorm_1.1-3        lubridate_1.8.0      xml2_1.3.3          
 [17] codetools_0.2-18     bridgesampling_1.1-2 shinythemes_1.2.0    bayesplot_1.9.0     
 [21] jsonlite_1.8.3       broom_1.0.1          dbplyr_2.2.1         shiny_1.7.3         
 [25] compiler_4.2.1       httr_1.4.4           emmeans_1.8.2        backports_1.4.1     
 [29] assertthat_0.2.1     Matrix_1.5-1         fastmap_1.1.0        gargle_1.2.1        
 [33] cli_3.4.1            later_1.3.0          htmltools_0.5.3      prettyunits_1.1.1   
 [37] tools_4.2.1          igraph_1.3.5         coda_0.19-4          gtable_0.3.1        
 [41] glue_1.6.2           reshape2_1.4.4       posterior_1.3.1      V8_4.2.1            
 [45] cellranger_1.1.0     vctrs_0.5.0          nlme_3.1-160         crosstalk_1.2.0     
 [49] tensorA_0.36.2       ps_1.7.2             rvest_1.0.3          mime_0.12           
 [53] miniUI_0.1.1.1       lifecycle_1.0.3      gtools_3.9.3         googlesheets4_1.0.1 
 [57] zoo_1.8-11           scales_1.2.1         colourpicker_1.2.0   hms_1.1.2           
 [61] promises_1.2.0.1     Brobdingnag_1.2-9    parallel_4.2.1       inline_0.3.19       
 [65] shinystan_2.6.0      curl_4.3.3           gridExtra_2.3        loo_2.5.1           
 [69] StanHeaders_2.26.13  stringi_1.7.8        dygraphs_1.1.1.6     checkmate_2.1.0     
 [73] boot_1.3-28          pkgbuild_1.3.1       rlang_1.0.6          pkgconfig_2.0.3     
 [77] matrixStats_0.62.0   distributional_0.3.1 lattice_0.20-45      rstantools_2.2.0    
 [81] htmlwidgets_1.5.4    tidyselect_1.2.0     processx_3.8.0       plyr_1.8.7          
 [85] magrittr_2.0.3       R6_2.5.1             generics_0.1.3       DBI_1.1.3           
 [89] pillar_1.8.1         haven_2.5.1          withr_2.5.0          xts_0.12.2          
 [93] abind_1.4-5          modelr_0.1.9         crayon_1.5.2         utf8_1.2.2          
 [97] tzdb_0.3.0           grid_4.2.1           readxl_1.4.1         callr_3.7.3         
[101] threejs_0.3.3        reprex_2.0.2         digest_0.6.30        xtable_1.8-4        
[105] httpuv_1.6.6         RcppParallel_5.1.5   stats4_4.2.1         munsell_0.5.0       
[109] shinyjs_2.1.0  

[jsocolar]

Hi @joshua-goldberg. Weird! I have a mac (different os though) on intel silicon running rstan 2.26.10 and I can run your example just fine. I don't have time to mess around with my rstan installation right now... Maybe the first thing to check is restarting R?

[joshua-goldberg]

Hi @jsocolar-

I'm pretty sure I had tried a restart before posting, but I opened a fresh instance R/Rstudio just now and get the same "parser failed badly" error with little hint of where the issue is emerging. Running via Terminal leads to the same error. I'm pretty sure I'm working with a fresh install of the dev version of rstan.

I suspect this is a Ventura-on-Intel thing, which is making me regret the snap decision to upgrade before more people from the Stan community vetted the transition. I avoided Catalina for a couple of years because of all the config issues, perhaps I should have done the same with Ventura.

I'm happy to help with troubleshooting when you have more time, but not proficient enough to develop a solution without guidance. I'll sit tight in the meantime.

[jsocolar]

I'm going to close this for the time being, since I don't think that the fix lies within flocker.