Unable to find include files when running through the worked example

[arranhamlet]

Hi,

I'm running into some issues running the ww-inference-model in the worked example you provided on the github repo. The error occurs at the fitting stage, with the function wwinference().

The error is:

The file functions/ar1.stan is found in the include path stated, and if I change the order of the include files in the stan model, it can't see any of them, so its not just an issue with that file. I've scoured github/stackoverflow/discourse.mc-stan.org, but not found a solution.

Steps I have tried:

1. Restarting R.
2. Checking I can run other models with cmdstanr, all good.
3. Checking to make sure that wwinference is pointing at the correct folder and the files specified are found.
4. Reinstalling wwinference, cmdstanr.
5. Double checking the file path and the folder permissions, making sure there are no issues.
6. Reinstalling R tools.

OS: Windows 10 I'm running cmdstan version 2.35.0, and my session info is:

R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8    LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.utf8    

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] lubridate_1.9.3        forcats_1.0.0          stringr_1.5.1          dplyr_1.1.4            purrr_1.0.2            readr_2.1.5            tidyr_1.3.1           
 [8] tibble_3.2.1           ggplot2_3.5.1          tidyverse_2.0.0        cmdstanr_0.8.1.9000    tidybayes_3.0.7        wwinference_0.0.0.9000 pacman_0.5.1          

loaded via a namespace (and not attached):
 [1] tensorA_0.36.2.1     ggdist_3.3.2         utf8_1.2.4           generics_0.1.3       stringi_1.8.4        lattice_0.22-6       svUnit_1.0.6        
 [8] hms_1.1.3            magrittr_2.0.3       grid_4.4.1           timechange_0.3.0     jsonlite_1.8.9       processx_3.8.4       backports_1.5.0     
[15] ps_1.8.0             fansi_1.0.6          scales_1.3.0         abind_1.4-8          cli_3.6.3            rlang_1.1.4          munsell_0.5.1       
[22] withr_3.0.1          tools_4.4.1          tzdb_0.4.0           checkmate_2.3.2      coda_0.19-4.1        colorspace_2.1-1     vctrs_0.6.5         
[29] posterior_1.6.0      R6_2.5.1             lifecycle_1.0.4      fs_1.6.4             pkgconfig_2.0.3      pillar_1.9.0         gtable_0.3.5        
[36] glue_1.7.0           Rcpp_1.0.13          xfun_0.47            tidyselect_1.2.1     rstudioapi_0.16.0    knitr_1.48           farver_2.1.2        
[43] labeling_0.4.3       RcppTOML_0.2.2       compiler_4.4.1       distributional_0.5.0 arrayhelpers_1.1-0  

[dylanhmorris]

This suggests it would be beneficial to run R CMD check in CI on macOS and Windows as well as Ubuntu. I will create an issue.

[kaitejohnson]

Update, I can't reproduce the error using R 4.4.1 on my Mac.

I would have guessed this is an issue with an update of cmdstan or cmdstanr where we have seen issues before when cmdstan gets stricter on say an extra space. But I am using the same version of cmdstanr and @arranhamlet is using cmdstan 2.35.0

[dylanhmorris]

@arranhamlet @kaitejohnson I suspect this may have to do with how non-Unix-style paths are handled

[kaitejohnson]

Hmm yeah in R this is why we have file.path("folder", "file") everywhere...

[kaitejohnson]

@arranhamlet Which version of R were you running before this broke if you remember? If it is a windows file system thing, I am surprised it ever worked!

[arranhamlet]

The same version, 4.4.1, very bizzare.

[seabbs]

This sounds like an annoying one and it also sounds like you've checked all the common sense things.

I've seen an issue like this with vscode where it has shown this error message but there is in fact no error (i.e everything runs). When I looked into it it was because cmdstan didn't like file systems with spaces in them (i.e here program files). That has since been resolved for me but perhaps it has resurfaced here?

I guess I would suggest/ask two things:

1. Is this an error in the sense you can't run the model or is it an error as I described where it printed an error message but everything was fine.
2. If you move the stan functions/model to a path that does not have a space does this work for you?

[arranhamlet]

1. It was an error in the sense that it couldn't run the model.
2. That was it, it was the space in "Program Files"! I've been able to compile the model and run it by moving it to a path that doesn't have a space.

Great work resolving this!

[kaitejohnson]

@arranhamlet

Interesting, so the problem is that the package gets installed in Program Files in Windows and then cmdstanr tried to include those paths in the include_paths and it breaks... wondering what we should do re README/documentation to warn about this issue.

Also super curious why reinstalling R drove this!

[seabbs]

Glad that got you unstuck!

wondering what we should do re README/documentation to warn about this issue.

So I think maybe this is worth pointing the cmdstan/cmdstanr folks at as I can imagine it coming up quite a bit in Windows and last time I checked they thought it was fixed.

[kaitejohnson]

@arranhamlet What was the precise solution you took? Did you reset the path that your R packages install to, or just this package? Would like to scope out an issue for cmdstanr/cmdstan and want to make sure I understand the solution you took.

[arranhamlet]

I moved the stan files from the package located at:

"C:\Program Files\R\R-4.4.1\library\wwinference\stan" to a new folder I created "C:\R"

and then updated the function wwinference::compile_model() to wwinference::compile_model(model_filepath = "C:/R/wwinference.stan", include_paths = "C:/R/")

[kaitejohnson]

Super helpful -- if you reinstall the package though, is it set up to install in C:\R?

[arranhamlet]

No, it would install it back in my default R library, which is found in "C:\Program Files\R\R-4.4.1\library".

I've not changed my library location, I just moved the stan files to above the Program Files directory to avoid the space in the name.

[kaitejohnson]

Hmm ok-- this is good to know as its not ideal for working off something that is in development as it will not update properly when you pull from git and reinstall.

I would think there is a way to specify that the library location is not in Program Files, which might be a better temporary solution for anything relying on cmdstan/ cmdstanr...

[dylanhmorris]

I would think there is a way to specify that the library location is not in Program Files, which might be a better temporary solution for anything relying on cmdstan/ cmdstanr...

I think we want to avoid having this package make any assumption about where it is installed. As @seabbs says, if cmdstanr doesn't currently play well with Windows paths containing spaces, that's something they'll want to know about and fix.

That said, perhaps time to reconsider depending on instantiate and compiling the package models at install time.

[kaitejohnson]

Yes, though this feels like a v2 issue!

[seabbs]

That said, perhaps time to reconsider depending on instantiate and compiling the package models at install time.

I think this uses cmdstanr and so will have the same error but just at install time

[dylanhmorris]

Depends on where instantiate places the source files to be compiled, no? Could potentially be somewhere more robust to this behavior than the user's default R library. Agree that the long-term solution is for cmdstanr to handle these paths!

[kaitejohnson]

@dylanhmorris Ok if we close this and continue discussion in #213 ?