search

rstudio / renv

renv: Project environments for R.
https://rstudio.github.io/renv/
MIT License
1.02k stars 155 forks source link

`renv` does not respect user repository configuration #1975

Open wibom opened 2 months ago

wibom commented 2 months ago

When in an renv-managed Rstudio project the user repository configuration is ignored.

The server I am working on does not have open internet access. I need to use a specific repos address, we'll call it "https://my.mirror.com/".

Global options

I have set the CRAN-mirror to use in the .Rprofile.

getOption("repos")
#> CRAN 
#> "https://my.mirror.com/"

Project options

When moving in to a project, not yet managed by renv, the global options are still respected:

renv::status()
#> This project does not appear to be using renv.
#> Use `renv::init()` to initialize the project.

getOption("repos")
#> CRAN 
#> "https://my.mirror.com/"

Initiating renv overrides the global options

When initiating renv the custom global option is replaced by cloud.r-project.org:

renv::init(bare = TRUE)
getOption("repos")   
#> CRAN 
#> "https://cloud.r-project.org"
# Clean up
renv::deactivate()
unlink("renv", recursive = TRUE)
unlink("renv.lock")

The repos argument to renv::init() also is not respected:

Passing in the CRAN mirror using the repos argument when initiating renv in the project has no effect:

renv::init(bare = TRUE, repos = "https://my.mirror.com/")
getOption("repos")  
#> CRAN 
#> "https://cloud.r-project.org"

[!NOTE] Unsurprisingly, all attempts so far results in a lock.file specifying ""https://cloud.r-project.org" as repository to use:

renv::snapshot()
str(renv:::renv_lockfile_read("renv.lock")$R)
#> List of 2
#> $ Version     : chr "4.4.1"
#> $ Repositories:List of 1
#> ..$ CRAN: chr "https://cloud.r-project.org"
# Clean up:
renv::deactivate()
unlink("renv", recursive = TRUE)
unlink("renv.lock")

Setting the repos option after initiating renv also is not respected:

Let's try setting the repos-option AFTER initiating renv:

renv::init(bare = TRUE)
options("repos" = c("CRAN" = "https://my.mirror.com/"))
getOption("repos")
#> CRAN 
#> "https://my.mirror.com/" 

renv::snapshot()
str(renv:::renv_lockfile_read("renv.lock")$R)
#> List of 2
#> $ Version     : chr "4.4.1"
#> $ Repositories:List of 1
#> ..$ CRAN: chr "https://my.mirror.com/"

Now it would appear all configurations are in place. However, I still don't think they are respected by renv. Consider the following:

# These settings are specified in the .Renviron file

Sys.getenv("RENV_CONFIG_REPOS_OVERRIDE")
#> [1] "https://my.mirror.com/"
Sys.getenv("RENV_CONFIG_CONNECT_TIMEOUT")
#> [1] "15"
Sys.getenv("RENV_CONFIG_CONNECT_RETRY")
#> [1] "0"

Let's see the time it takes to install a package using renv::install(): 

tictoc::tic()
renv::install("digest", prompt = FALSE)
#> # Installing packages --------------------------------------------------------
#> - Installing digest ...                         OK [copied from cache in 0.27s]
#> Successfully installed 1 package in 0.28 seconds.
tictoc::toc()
#> 15.83 sec elapsed

# Use a different RENV_CONFIG_CONNECT_TIMEOUT
Sys.setenv(RENV_CONFIG_CONNECT_TIMEOUT = 25)
tictoc::tic()
renv::install("digest", prompt = FALSE)
#> # Installing packages --------------------------------------------------------
#> - Installing digest ...                         OK [copied from cache in 0.26s]
#> Successfully installed 1 package in 0.32 seconds.
tictoc::toc()
#> 25.39 sec elapsed

I don't know what happens during the time it takes renv::install() to install the package, but the fact that we can control how long it takes by changing the RENV_CONFIG_CONNECT_TIMEOUT-setting indicates that renv is attempting to connect to somewhere other than the repository specified by getOption("repos")/Sys.getenv("RENV_CONFIG_REPOS_OVERRIDE").

renv::diagnostics()

Diagnostics Report [renv 1.0.7]
===============================

# Session Info ---------------------------------------------------------------
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows Server 2019 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=Swedish_Sweden.1252  LC_CTYPE=Swedish_Sweden.1252   
[3] LC_MONETARY=Swedish_Sweden.1252 LC_NUMERIC=C                   
[5] LC_TIME=Swedish_Sweden.1252    

time zone: Europe/Stockholm
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.4.1 tictoc_1.2.1   tools_4.4.1    renv_1.0.7    

# Project --------------------------------------------------------------------
Project path: "C:/Users/xxx/Desktop/renv_test7"

# Status ---------------------------------------------------------------------
No issues found -- the project is in a consistent state.

# Packages -------------------------------------------------------------------
            Library Source Lockfile Source Path Dependency
KernSmooth  2.23-24   CRAN     <NA>   <NA>  [2]       <NA>
MASS       7.3-60.2   CRAN     <NA>   <NA>  [2]       <NA>
Matrix        1.7-0   CRAN     <NA>   <NA>  [2]       <NA>
boot         1.3-30   CRAN     <NA>   <NA>  [2]       <NA>
class        7.3-22   CRAN     <NA>   <NA>  [2]       <NA>
cluster       2.1.6   CRAN     <NA>   <NA>  [2]       <NA>
codetools    0.2-20   CRAN     <NA>   <NA>  [2]       <NA>
digest       0.6.37   CRAN     <NA>   <NA>  [1]       <NA>
foreign      0.8-86   CRAN     <NA>   <NA>  [2]       <NA>
lattice      0.22-6   CRAN     <NA>   <NA>  [2]       <NA>
mgcv          1.9-1   CRAN     <NA>   <NA>  [2]       <NA>
nlme        3.1-164   CRAN     <NA>   <NA>  [2]       <NA>
nnet         7.3-19   CRAN     <NA>   <NA>  [2]       <NA>
renv          1.0.7   CRAN    1.0.7   CRAN  [1]     direct
rpart        4.1.23   CRAN     <NA>   <NA>  [2]       <NA>
spatial      7.3-17   CRAN     <NA>   <NA>  [2]       <NA>
survival      3.6-4   CRAN     <NA>   <NA>  [2]       <NA>
tictoc        1.2.1   CRAN     <NA>   <NA>  [1]       <NA>
utils          <NA>   <NA>     <NA>   <NA>  [2]   indirect

[1]: C:/Users/xxx/Desktop/renv_test7/renv/library/windows/R-4.4/x86_64-w64-mingw32              
[2]: C:/Users/xxx/AppData/Local/R/cache/R/renv/sandbox/windows/R-4.4/x86_64-w64-mingw32/e0da0d43

# ABI ------------------------------------------------------------------------
- ABI conflict checks are not available on Windows.

# User Profile ---------------------------------------------------------------
[1] Source  Package Require Version Dev    
<0 rows> (or 0-length row.names)

# Settings -------------------------------------------------------------------
List of 13
 $ bioconductor.version     : NULL
 $ external.libraries       : chr(0) 
 $ ignored.packages         : chr(0) 
 $ package.dependency.fields: chr [1:3] "Imports" "Depends" "LinkingTo"
 $ ppm.enabled              : NULL
 $ ppm.ignored.urls         : chr(0) 
 $ r.version                : NULL
 $ snapshot.type            : chr "implicit"
 $ use.cache                : logi TRUE
 $ vcs.ignore.cellar        : logi TRUE
 $ vcs.ignore.library       : logi TRUE
 $ vcs.ignore.local         : logi TRUE
 $ vcs.manage.ignores       : logi TRUE

# Options --------------------------------------------------------------------
List of 8
 $ defaultPackages                     : chr [1:6] "datasets" "utils" "grDevices" "graphics" ...
 $ download.file.method                : chr "libcurl"
 $ download.file.extra                 : NULL
 $ install.packages.compile.from.source: chr "interactive"
 $ pkgType                             : chr "both"
 $ repos                               : Named chr "https://my.mirror.com/"
  ..- attr(*, "names")= chr "CRAN"
 $ renv.consent                        : logi TRUE
 $ renv.verbose                        : logi TRUE

# Environment Variables ------------------------------------------------------
HOME                        = C:\Users\xxx\Documents
LANG                        = <NA>
MAKE                        = <NA>
R_LIBS                      = <NA>
R_LIBS_SITE                 = C:/PROGRA~1/R/R-44~1.1/site-library
R_LIBS_USER                 = C:/Users/xxx/Desktop/renv_test7/renv/library/windows/R-4.4/x86_64-w64-mingw32
RENV_CACHE_MODE             = 777
RENV_CONFIG_CONNECT_RETRY   = 0
RENV_CONFIG_CONNECT_TIMEOUT = 25
RENV_CONFIG_REPOS_OVERRIDE  = https://my.mirror.com/
RENV_DEFAULT_R_ENVIRON      = <NA>
RENV_DEFAULT_R_ENVIRON_USER = <NA>
RENV_DEFAULT_R_LIBS         = <NA>
RENV_DEFAULT_R_LIBS_SITE    = C:/PROGRA~1/R/R-44~1.1/site-library
RENV_DEFAULT_R_LIBS_USER    = C:\Users\xxx\AppData\Local/R/win-library/4.4
RENV_DEFAULT_R_PROFILE      = <NA>
RENV_DEFAULT_R_PROFILE_USER = <NA>
RENV_PATHS_CACHE            = D:/REF/.renv
RENV_PATHS_CELLAR           = D:/REF/.renv/cellar
RENV_PROJECT                = C:/Users/xxx/Desktop/renv_test7

# PATH -----------------------------------------------------------------------
- C:\rtools40\usr\bin
- c:\rtools44\x86_64-w64-mingw32.static.posix\bin
- c:\rtools44\usr\bin
- C:\rtools40\usr\bin
- C:\rtools40\usr\bin
- C:\Program Files\R\R-4.4.1\bin\x64
- C:\WINDOWS\system32
- C:\WINDOWS
- C:\WINDOWS\System32\Wbem
- C:\WINDOWS\System32\WindowsPowerShell\v1.0\
- C:\WINDOWS\System32\OpenSSH\
- C:\Program Files\\SUT\bin
- C:\Program Files\PuTTY\
- C:\Program Files\AzureConnectedMachineAgent\
- C:\Users\xxx\AppData\Local\Microsoft\WindowsApps
- C:\Users\xxx\Ubuntu
- C:\Users\xxx\apps_portable\RStudio-2024.04.2-764\resources\app\bin\quarto\bin
- C:\Users\xxx\apps_portable\RStudio-2024.04.2-764\resources\app\bin\postback

# Cache ----------------------------------------------------------------------
There are a total of 2 packages installed in the renv cache.
Cache path: "D:/REF/.renv/v5/windows/R-4.4/x86_64-w64-mingw32"
kevinushey commented 2 months ago

I wonder if this is possibly an RStudio issue -- do you see the same behavior if you use renv outside of RStudio; e.g. in RGui on Windows?

wibom commented 2 months ago

Thanks for your input!!!

It seems you are right about Rstudio being the culprit for ignoring the global "repos" option. I'll see if I can find an appropriate repo to post that part of the issue in...

However, also when running directly in the Rgui for windows, the time it takes to grab a package is controlled by the RENV_CONFIG_CONNECT_TIMEOUT-setting. That seems a bit strange.

Here are some quick tests from Rgui on Windows:

# Global options
dir.create("test-renv-1")
setwd("test-renv-1")
renv::status()
#> This project does not appear to be using renv.
#> Use `renv::init()` to initialize the project.
getOption("repos")
#>                  CRAN 
#> "https://my.mirror.com/" 

# Create renv-managed project
renv::init(bare = TRUE)
#> - renv activated -- please restart the R session.

### RESTART

setwd("test-renv-1")
source("renv/activate.R")
renv::snapshot()
renv::status()
#> No issues found -- the project is in a consistent state.

getOption("repos")
#>                  CRAN 
#> "https://my.mirror.com/"

So far so good. The global options are respected by renv.

However, the actual time it takes to get a package can still be controlled by the RENV_CONFIG_CONNECT_TIMEOUT-setting. 

renv::install("tictoc")

Sys.getenv("RENV_CONFIG_CONNECT_TIMEOUT")
#> [1] "15"
tictoc::tic()
renv::install("digest", prompt = FALSE)
tictoc::toc()
#> 15.74 sec elapsed

Sys.setenv(RENV_CONFIG_CONNECT_TIMEOUT = 25)
tictoc::tic()
renv::install("digest", prompt = FALSE)
#> The following package(s) will be installed:
#> - digest [0.6.37]
#> These packages will be installed into "~/test-renv-1/renv/library/windows/R-4.4/x86_64-w64-mingw32".

#> # Installing packages --------------------------------------------------------
#> - Installing digest ...                         OK [copied from cache in 0.37s]
#> Successfully installed 1 package in 0.43 seconds.
tictoc::toc()
#> 25.75 sec elapsed

Could this a bug somewhere? Or have I messed up the configurations somewhere 🤔?

wibom commented 2 months ago

Resolved - not a bug, just poor configuration! 🙈

I found the reason that the RENV_CONFIG_CONNECT_TIMEOUT-setting controls the time it takes renv::install() to install a package.

Tracing the call stack of renv::install, I found this call some way down the stack: renv:::renv_p3m_database_refresh(explicit = FALSE). On our locked down server, the call results in # Error: error downloading 'https://rstudio-buildtools.s3.amazonaws.com/renv/package-manager/packages.rds' [error code 28].

The ERROR is handled within renv:::renv_available_packages_latest() and hidden from the user.

Now that I know what is going on, it's easy to fix by setting renv.config.ppm.enabled and/or renv.config.ppm.default to FALSE (both defaults to TRUE):

Summary

  1. Rstudio causes renv::init() to override any customization of the global repos-option and replace it by cloud.r-project.org.
  2. Users with limited internet access may wish to opt out of Posit Package Manager integration by configuring renv.config.ppm.enabled.