HenrikBengtsson / CBI-software

A Scientific Software Stack for HPC (CentOS oriented)
https://wynton.ucsf.edu/hpc/software/software-repositories.html
5 stars 2 forks source link

R: Make renv use file paths that are specific to also the Linux distribution #118

Closed HenrikBengtsson closed 6 months ago

HenrikBengtsson commented 6 months ago

When migrating from CentOS 7 to Rocky 8, users relying on renv environments will end up loading R packages installed for CentOS 7.

From help("paths", package = "renv"):


Sharing state across operating systems

If you need to share the same cache with multiple different Linux operating systems, you may want to set the RENV_PATHS_PREFIX environment variable to help disambiguate the paths used on Linux. For example, setting RENV_PATHS_PREFIX = "ubuntu-bionic" would instruct renv to construct a cache path like:

/mnt/shared/renv/cache/v2/ubuntu-bionic/R-3.5/x86_64-pc-linux-gnu

HenrikBengtsson commented 6 months ago

Even better, from help("paths", package = "renv"):

Starting from ⁠renv 0.13.0⁠ [edit: released on 2021-02-24], you can also instruct renv to auto-generate an OS-specific component to include as part of library and cache paths, by setting the environment variable:

RENV_PATHS_PREFIX_AUTO = TRUE

The prefix will be constructed based on fields within the system's ⁠/etc/os-release⁠ file.


So, setting

export RENV_PATHS_PREFIX_AUTO=TRUE

seems to be the simplest and most agile solution.

One could imagine why this is not the default. It's indeed been proposed (https://github.com/rstudio/renv/issues/1211) and is part of the renv 1.1.0 milestone. FWIW, current version of renv is 1.0.3 (2023-09-19).

HenrikBengtsson commented 6 months ago

Deploy this to all versions of R on Wynton:

mapfile -t versions < <(MODULEPATH=/wynton/home/cbi/shared/modulefiles/CBI/_rocky8 module --terse spider r 2>&1 | sed 's|r/||' | sed 's|-gcc.*||' | sort -u)
for version in "${versions[@]}"; do make module VERSION="${version}"; done
HenrikBengtsson commented 6 months ago

OTH, using RENV_PATHS_PREFIX_AUTO=TRUE might be too specific, e.g.

> .libPaths()
[1] "/wynton/home/bengtsson/hb-test/foo/renv/library/linux-rocky-8.8/R-4.3/x86_64-pc-linux-gnu"              
[2] "/wynton/home/bengtsson/hb-test/.cache/R/renv/sandbox/linux-rocky-8.8/R-4.3/x86_64-pc-linux-gnu/78481ac5"

This means that Rocky 8.8 and Rocky 8.9 will use different paths. It's probably sufficient to make it specific to Rocky 8.

OTH OTH, if renv plans on making this the new default, we might equally well use it already now. So, it comes down to which horse we want to bet on.

HenrikBengtsson commented 6 months ago

Done.