"sh: clang++ command not found" in renv project

[mvuorre]

Hi, I have an issue using Stan (via brms) inside a project that uses renv. To reproduce the problem, I start a fresh project, click the use renv button in rstudio, and do the following:

# This is in Untitled.R file
# install.packages("brms")  # Run once
library(brms)
## Save Untitled.R
# renv::snapshot()  # Run once
## Restart R
brm(mpg ~ cyl, data = mtcars, chains = 1)

The above line results in the following errors

sh: clang++ -mmacosx-version-min=10.13: command not found
Compiling the C++ model
sh: clang++ -mmacosx-version-min=10.13: command not found

If I run the exact same code, without renv, outside a project (or in another project without renv), everything works fine.

sessionInfo()

R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.5

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] brms_2.13.0 Rcpp_1.0.5 

loaded via a namespace (and not attached):
 [1] Brobdingnag_1.2-6    jsonlite_1.7.0       gtools_3.8.2         StanHeaders_2.21.0-5
 [5] RcppParallel_5.0.2   threejs_0.3.3        shiny_1.5.0          assertthat_0.2.1    
 [9] stats4_4.0.2         renv_0.11.0          backports_1.1.8      pillar_1.4.5        
[13] lattice_0.20-41      glue_1.4.1           digest_0.6.25        promises_1.1.1      
[17] colorspace_1.4-1     htmltools_0.5.0      httpuv_1.5.4         Matrix_1.2-18       
[21] plyr_1.8.6           dygraphs_1.1.1.6     pkgconfig_2.0.3      rstan_2.21.1        
[25] purrr_0.3.4          xtable_1.8-4         mvtnorm_1.1-1        scales_1.1.1        
[29] processx_3.4.3       later_1.1.0.1        tibble_3.0.2         bayesplot_1.7.2     
[33] generics_0.0.2       ggplot2_3.3.2        ellipsis_0.3.1       DT_0.14             
[37] shinyjs_1.1          cli_2.0.2            magrittr_1.5         crayon_1.3.4        
[41] mime_0.9             ps_1.3.3             fansi_0.4.1          nlme_3.1-148        
[45] xts_0.12-0           pkgbuild_1.0.8       colourpicker_1.0     prettyunits_1.1.1   
[49] rsconnect_0.8.16     tools_4.0.2          loo_2.3.0            lifecycle_0.2.0     
[53] matrixStats_0.56.0   stringr_1.4.0        V8_3.2.0             munsell_0.5.0       
[57] callr_3.4.3          packrat_0.5.0        compiler_4.0.2       rlang_0.4.7         
[61] grid_4.0.2           ggridges_0.5.2       rstudioapi_0.11      htmlwidgets_1.5.1   
[65] crosstalk_1.1.0.1    igraph_1.2.5         miniUI_0.1.1.1       base64enc_0.1-3     
[69] codetools_0.2-16     gtable_0.3.0         curl_4.3             inline_0.3.15       
[73] abind_1.4-5          markdown_1.1         reshape2_1.4.4       R6_2.4.1            
[77] gridExtra_2.3        rstantools_2.1.1     zoo_1.8-8            bridgesampling_1.0-0
[81] dplyr_1.0.0          fastmap_1.0.1        shinystan_2.5.0      shinythemes_1.1.2   
[85] stringi_1.4.6        parallel_4.0.2       vctrs_0.3.1          tidyselect_1.1.0    
[89] coda_0.19-3 

[niklasjohannes]

With the same procedure, but on Windows, the model runs, but I get a warning:

Warning messages:
1: In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\nikla\AppData\Local\Temp\Rtmpwh59w2\file4f644cb73474.stan', reason 'No such file or directory'
2: In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\nikla\AppData\Local\Temp\Rtmpwh59w2\file4f642e667c3e.stan', reason 'No such file or directory'

I had brms installed before installing it in the project, so renv links to it in the project library.

sessionInfo()

R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] brms_2.13.0 Rcpp_1.0.5 

loaded via a namespace (and not attached):
 [1] Brobdingnag_1.2-6    jsonlite_1.7.0       gtools_3.8.2         StanHeaders_2.21.0-5 RcppParallel_5.0.2   threejs_0.3.3       
 [7] shiny_1.5.0          assertthat_0.2.1     stats4_3.6.3         renv_0.10.0          backports_1.1.8      pillar_1.4.5        
[13] lattice_0.20-38      glue_1.4.1           digest_0.6.25        promises_1.1.1       colorspace_1.4-1     htmltools_0.5.0     
[19] httpuv_1.5.4         Matrix_1.2-18        plyr_1.8.6           dygraphs_1.1.1.6     pkgconfig_2.0.3      rstan_2.21.1        
[25] purrr_0.3.4          xtable_1.8-4         mvtnorm_1.1-1        scales_1.1.1         processx_3.4.3       later_1.1.0.1       
[31] tibble_3.0.2         bayesplot_1.7.2      generics_0.0.2       ggplot2_3.3.2        ellipsis_0.3.1       DT_0.14             
[37] withr_2.2.0          shinyjs_1.1          cli_2.0.2            magrittr_1.5         crayon_1.3.4         mime_0.9            
[43] ps_1.3.3             fansi_0.4.1          nlme_3.1-144         xts_0.12-0           pkgbuild_1.0.8       colourpicker_1.0    
[49] prettyunits_1.1.1    rsconnect_0.8.16     tools_3.6.3          loo_2.3.0            lifecycle_0.2.0      matrixStats_0.56.0  
[55] stringr_1.4.0        V8_3.2.0             munsell_0.5.0        callr_3.4.3          packrat_0.5.0        compiler_3.6.3      
[61] rlang_0.4.7          grid_3.6.3           ggridges_0.5.2       rstudioapi_0.11      htmlwidgets_1.5.1    crosstalk_1.1.0.1   
[67] igraph_1.2.5         miniUI_0.1.1.1       base64enc_0.1-3      codetools_0.2-16     gtable_0.3.0         curl_4.3            
[73] inline_0.3.15        abind_1.4-5          markdown_1.1         reshape2_1.4.4       R6_2.4.1             gridExtra_2.3       
[79] rstantools_2.1.1     zoo_1.8-8            bridgesampling_1.0-0 dplyr_1.0.0          fastmap_1.0.1        shinystan_2.5.0     
[85] shinythemes_1.1.2    stringi_1.4.6        parallel_3.6.3       vctrs_0.3.1          tidyselect_1.1.0     coda_0.19-3    

I also don't get those warnings if I run the same model in a project without renv.

[kevinushey]

Is clang++ on the PATH? What is the output of Sys.which("clang++") in each case?

Do you do some work in your ~/.Rprofile to set up compiler paths? Note that renv doesn't run the user .Rprofile by default, but you could edit the file at ~/.Renviron and adding:

RENV_CONFIG_USER_PROFILE = TRUE

[mvuorre]

Thanks for the pointers.

> Sys.which("clang++")
           clang++ 
"/usr/bin/clang++" 

There's no compiler paths in ~/.Rprofile (there's a fix for a bug in parallel processing):

> readLines("~/.Rprofile")
[1] "## WORKAROUND: https://github.com/rstudio/rstudio/issues/6692"                           
[2] "## Revert to 'sequential' setup of PSOCK cluster in RStudio Console on macOS and R 4.0.0"
[3] "if (Sys.getenv(\"RSTUDIO\") == \"1\" && !nzchar(Sys.getenv(\"RSTUDIO_TERM\")) && "       
[4] "    Sys.info()[\"sysname\"] == \"Darwin\") {"                                            
[5] "  parallel:::setDefaultClusterOptions(setup_strategy = \"sequential\")"                  
[6] "}"

> readLines("~/.Renviron")
[1] "TWITTER_PAT=<removed>"

I also tried to put your suggested line in here and retried, but to no effect (unsurprisingly).

I do have a Makevars file as per RStan docs, but that's probably not related:

> readLines("~/.R/Makevars")
[1] ""                                               
[2] ""                                               
[3] ""                                               
[4] "CXX14FLAGS=-O3 -march=native -mtune=native"     
[5] "CXX14FLAGS += -arch x86_64 -ftemplate-depth-256"

[kevinushey]

Let me try to reproduce and see if I can learn more...

[kevinushey]

Works on my machine. :-/

You might need to try something like:

debug(system)
debug(system2)

to see where Stan is invoking the C++ compiler, and see if you can figure out what's causing the issue here.

[mvuorre]

Thanks, I'll keep digging, and will post here if I find a solution.

[harrismcgehee]

I believe I have a similar issue.

The path C++ compiler is below the version required for odbc (https://github.com/r-dbi/odbc/issues/377). So, in ~/.R/Makevars, I set the paths:

CC=/path/to/gcc
CXX=/path/to/g++
CXX14=/path/to/g++

Now, when renv tries to install odbc, it fails as if it doesn't find the Makevars file.

in fact, it tells me so?

* installing *source* package ‘odbc’ ...
** package ‘odbc’ successfully unpacked and MD5 sums checked
make: Makevars: No such file or directory
make: *** No rule to make target `Makevars'.  Stop.
Found odbc_config cflags and libs!
PKG_CFLAGS=-DHAVE_UNISTD_H -DHAVE_PWD_H -DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG_INT=8 -I/usr/include 
PKG_LIBS=-L/usr/lib64 -lodbc
** libs
(cd cctz && \
make libcctz.a CC="/path/to/gcc" CXX="g++ -std=gnu++11" AR="ar" ARFLAGS=rv CXXPICFLAGS="-fpic")

This stands out to me: CXX="g++ -std=gnu++11"

[kevinushey]

Are you able to double-check what the output of:

Sys.getenv("R_MAKEVARS_USER")
path.expand("~/.R/Makevars")
file.exists("~/.R/Makevars")

are?

[harrismcgehee]

Sys.getenv("R_MAKEVARS_USER")
#[1] ""
path.expand("~/.R/Makevars")
#[1] "/home/<user>/.R/Makevars"
file.exists("~/.R/Makevars")
#[1] TRUE

I attempted setting the environment variable before attempting an install of odbc.

Sys.setenv("R_MAKEVARS_USER" = "~/.R/Makevars")
renv::install("odbc")
# ...
# Installing odbc [1.2.3] ...
# FAILED
# Error installing package 'odbc':
#     ================================
#     
#     * installing *source* package ‘odbc’ ...
# ** package ‘odbc’ successfully unpacked and MD5 sums checked
# make: Makevars: No such file or directory
# make: *** No rule to make target `Makevars'.  Stop.
# Found odbc_config cflags and libs!
# PKG_CFLAGS=-DHAVE_UNISTD_H -DHAVE_PWD_H -DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG_INT=8 -I/usr/include 
# PKG_LIBS=-L/usr/lib64 -lodbc

Sys.getenv("R_MAKEVARS_USER")
# [1] "~/.R/Makevars"

Do I need to set R_MAKEVARS_USER earlier?

[hadley]

@harrismcgehee Is this still a problem for you?

[harrismcgehee]

It is not still a problem at this time.

I am still setting the paths in my ~/.R/Makevars file.