search

UCL-ARC / hpc-spack

Solutions - HPC's Spack config
MIT License
1 stars 2 forks source link

How should we bundle R and Bioconductor? #16

Open balston opened 1 year ago

balston commented 1 year ago

We need to decide how we bundle R plus Bioconductor and all the additional R packages we use on the clusters. We need an equivalent of doing:

module -f unload compilers mpi gcc-libs
module load beta-modules
module load r/r-4.2.0_bc-3.15
balston commented 1 year ago

Running:

spack info r

shows:

AutotoolsPackage:   r

Description:
    R is 'GNU S', a freely available language and environment for
    statistical computing and graphics which provides a wide variety of
    statistical and graphical techniques: linear and nonlinear modelling,
    statistical tests, time series analysis, classification, clustering,
    etc. Please consult the R project homepage for further information.

Homepage: https://www.r-project.org

Preferred version:
    4.2.2    https://cloud.r-project.org/src/base/R-4/R-4.2.2.tar.gz

Safe versions:
    4.2.2    https://cloud.r-project.org/src/base/R-4/R-4.2.2.tar.gz
    4.2.1    https://cloud.r-project.org/src/base/R-4/R-4.2.1.tar.gz
    4.2.0    https://cloud.r-project.org/src/base/R-4/R-4.2.0.tar.gz
    4.1.3    https://cloud.r-project.org/src/base/R-4/R-4.1.3.tar.gz
    4.1.2    https://cloud.r-project.org/src/base/R-4/R-4.1.2.tar.gz
    4.1.1    https://cloud.r-project.org/src/base/R-4/R-4.1.1.tar.gz
    4.1.0    https://cloud.r-project.org/src/base/R-4/R-4.1.0.tar.gz
    4.0.5    https://cloud.r-project.org/src/base/R-4/R-4.0.5.tar.gz
    4.0.4    https://cloud.r-project.org/src/base/R-4/R-4.0.4.tar.gz
    4.0.3    https://cloud.r-project.org/src/base/R-4/R-4.0.3.tar.gz
    4.0.2    https://cloud.r-project.org/src/base/R-4/R-4.0.2.tar.gz
    4.0.1    https://cloud.r-project.org/src/base/R-4/R-4.0.1.tar.gz
    4.0.0    https://cloud.r-project.org/src/base/R-4/R-4.0.0.tar.gz
    3.6.3    https://cloud.r-project.org/src/base/R-3/R-3.6.3.tar.gz
    3.6.2    https://cloud.r-project.org/src/base/R-3/R-3.6.2.tar.gz
    3.6.1    https://cloud.r-project.org/src/base/R-3/R-3.6.1.tar.gz
    3.6.0    https://cloud.r-project.org/src/base/R-3/R-3.6.0.tar.gz
    3.5.3    https://cloud.r-project.org/src/base/R-3/R-3.5.3.tar.gz
    3.5.2    https://cloud.r-project.org/src/base/R-3/R-3.5.2.tar.gz
    3.5.1    https://cloud.r-project.org/src/base/R-3/R-3.5.1.tar.gz
    3.5.0    https://cloud.r-project.org/src/base/R-3/R-3.5.0.tar.gz
    3.4.4    https://cloud.r-project.org/src/base/R-3/R-3.4.4.tar.gz
    3.4.3    https://cloud.r-project.org/src/base/R-3/R-3.4.3.tar.gz
    3.4.2    https://cloud.r-project.org/src/base/R-3/R-3.4.2.tar.gz
    3.4.1    https://cloud.r-project.org/src/base/R-3/R-3.4.1.tar.gz
    3.4.0    https://cloud.r-project.org/src/base/R-3/R-3.4.0.tar.gz
    3.3.3    https://cloud.r-project.org/src/base/R-3/R-3.3.3.tar.gz
    3.3.2    https://cloud.r-project.org/src/base/R-3/R-3.3.2.tar.gz
    3.3.1    https://cloud.r-project.org/src/base/R-3/R-3.3.1.tar.gz
    3.3.0    https://cloud.r-project.org/src/base/R-3/R-3.3.0.tar.gz
    3.2.5    https://cloud.r-project.org/src/base/R-3/R-3.2.5.tar.gz
    3.2.3    https://cloud.r-project.org/src/base/R-3/R-3.2.3.tar.gz
    3.2.2    https://cloud.r-project.org/src/base/R-3/R-3.2.2.tar.gz
    3.2.1    https://cloud.r-project.org/src/base/R-3/R-3.2.1.tar.gz
    3.2.0    https://cloud.r-project.org/src/base/R-3/R-3.2.0.tar.gz
    3.1.3    https://cloud.r-project.org/src/base/R-3/R-3.1.3.tar.gz
    3.1.2    https://cloud.r-project.org/src/base/R-3/R-3.1.2.tar.gz

Deprecated versions:
    None

Variants:
    Name [Default]              When    Allowed values    Description
    ========================    ====    ==============    ============================

    X [off]                     --      on, off           Enable X11 support (TCLTK,
                                                          PNG, JPEG, TIFF, CAIRO)
    build_system [autotools]    --      autotools         Build systems supported by
                                                          the package
    external-lapack [off]       --      on, off           Links to externally
                                                          installed BLAS/LAPACK
    memory_profiling [off]      --      on, off           Enable memory profiling
    rmath [off]                 --      on, off           Build standalone Rmath
                                                          library

Build Dependencies:
    blas   curl       icu4c  lapack   libx11  ncurses  pcre2     tk     zlib
    bzip2  gnuconfig  java   libpng   libxmu  pango    readline  which
    cairo  harfbuzz   jpeg   libtiff  libxt   pcre     texinfo   xz

Link Dependencies:
    blas   curl      java    libpng   libxmu   pango  readline  zlib
    bzip2  harfbuzz  jpeg    libtiff  libxt    pcre   tk
    cairo  icu4c     lapack  libx11   ncurses  pcre2  xz

Run Dependencies:
    which
balston commented 1 year ago

From the above we probably need to install OpenBLAS first to be able to use the external-lapack option which is needed to match our build scripts method.

spack info openblas
MakefilePackage:   openblas

Description:
    OpenBLAS: An optimized BLAS library

Homepage: https://www.openblas.net

Preferred version:
    0.3.21     https://github.com/xianyi/OpenBLAS/archive/v0.3.21.tar.gz

Safe versions:
    develop    [git] https://github.com/xianyi/OpenBLAS.git on branch develop
    0.3.21     https://github.com/xianyi/OpenBLAS/archive/v0.3.21.tar.gz
    0.3.20     https://github.com/xianyi/OpenBLAS/archive/v0.3.20.tar.gz
    0.3.19     https://github.com/xianyi/OpenBLAS/archive/v0.3.19.tar.gz
    0.3.18     https://github.com/xianyi/OpenBLAS/archive/v0.3.18.tar.gz
    0.3.17     https://github.com/xianyi/OpenBLAS/archive/v0.3.17.tar.gz
    0.3.16     https://github.com/xianyi/OpenBLAS/archive/v0.3.16.tar.gz
    0.3.15     https://github.com/xianyi/OpenBLAS/archive/v0.3.15.tar.gz
    0.3.14     https://github.com/xianyi/OpenBLAS/archive/v0.3.14.tar.gz
    0.3.13     https://github.com/xianyi/OpenBLAS/archive/v0.3.13.tar.gz
    0.3.12     https://github.com/xianyi/OpenBLAS/archive/v0.3.12.tar.gz
    0.3.11     https://github.com/xianyi/OpenBLAS/archive/v0.3.11.tar.gz
    0.3.10     https://github.com/xianyi/OpenBLAS/archive/v0.3.10.tar.gz
    0.3.9      https://github.com/xianyi/OpenBLAS/archive/v0.3.9.tar.gz
    0.3.8      https://github.com/xianyi/OpenBLAS/archive/v0.3.8.tar.gz
    0.3.7      https://github.com/xianyi/OpenBLAS/archive/v0.3.7.tar.gz
    0.3.6      https://github.com/xianyi/OpenBLAS/archive/v0.3.6.tar.gz
    0.3.5      https://github.com/xianyi/OpenBLAS/archive/v0.3.5.tar.gz
    0.3.4      https://github.com/xianyi/OpenBLAS/archive/v0.3.4.tar.gz
    0.3.3      https://github.com/xianyi/OpenBLAS/archive/v0.3.3.tar.gz
    0.3.2      https://github.com/xianyi/OpenBLAS/archive/v0.3.2.tar.gz
    0.3.1      https://github.com/xianyi/OpenBLAS/archive/v0.3.1.tar.gz
    0.3.0      https://github.com/xianyi/OpenBLAS/archive/v0.3.0.tar.gz
    0.2.20     https://github.com/xianyi/OpenBLAS/archive/v0.2.20.tar.gz
    0.2.19     https://github.com/xianyi/OpenBLAS/archive/v0.2.19.tar.gz
    0.2.18     https://github.com/xianyi/OpenBLAS/archive/v0.2.18.tar.gz
    0.2.17     https://github.com/xianyi/OpenBLAS/archive/v0.2.17.tar.gz
    0.2.16     https://github.com/xianyi/OpenBLAS/archive/v0.2.16.tar.gz
    0.2.15     https://github.com/xianyi/OpenBLAS/archive/v0.2.15.tar.gz

Deprecated versions:
    None

Variants:
    Name [Default]             When          Allowed values          Description
    =======================    ==========    ====================    =======================

    bignuma [off]              --            on, off                 Enable experimental
                                                                     support for up to 1024
                                                                     CPUs/Cores and 128 numa
                                                                     nodes
    build_system [makefile]    --            makefile                Build systems supported
                                                                     by the package
    consistent_fpcsr [off]     --            on, off                 Synchronize FP CSR
                                                                     between threads
                                                                     (x86/x86_64 only)
    fortran [True]             [@0.3.21:]    on, off                 w/o a Fortran compiler,
                                                                     OpenBLAS will build an
                                                                     f2c-converted LAPACK
    ilp64 [off]                --            on, off                 Force 64-bit Fortran
                                                                     native integers
    locking [on]               --            on, off                 Build with thread
                                                                     safety
    pic [on]                   --            on, off                 Build position
                                                                     independent code
    shared [on]                --            on, off                 Build shared libraries
    symbol_suffix [none]       --                                    Set a symbol suffix
    threads [none]             --            pthreads, openmp,       Multithreading support
                                             none

Build Dependencies:
    perl

Link Dependencies:
    None

Run Dependencies:
    None
balston commented 1 year ago

I now have three OpenBLAS builds. Each one built using a command like:

spack install -v openblas threads=openmp %gcc@12.2.0 2>&1 | tee OpenBLAS-openmp-build.log
spack find --long --variants openblas
-- linux-rhel7-skylake_avx512 / gcc@12.2.0 ----------------------
qj54sj4 openblas@0.3.21~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile patches=d3d9b15 symbol_suffix=none threads=none
quoo6tc openblas@0.3.21~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile patches=d3d9b15 symbol_suffix=none threads=openmp
bonrh5e openblas@0.3.21~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile patches=d3d9b15 symbol_suffix=none threads=pthreads
==> 3 installed packages
balston commented 1 year ago

From reading the Spack docs about their R install method, Spack sets the:

--without-recommended-packages

option so the recommended packages that the R developers expect to be installed by default are not. Not sure at the moment how to work with this.

This is the current config step for R builds on the clusters:

./configure -C --prefix=${INSTALL_PREFIX} --with-recommended-packages --with-x \
      --with-readline=yes --enable-R-shlib \
      --with-blas="-L /shared/ucl/apps/openblas/0.3.13-serial/gnu-10.2.0/lib -lopenblas" \
      --enable-BLAS-shlib
balston commented 1 year ago

As something to compare against, I've built the base version of R 4.2.2 (the current released version) using our current build-scripts method. This includes the recommended packages so we get the following. before adding our set of extra packages:

 ls -l /shared/ucl/apps/R/R-4.2.2-OpenBLAS/lib64/R/library/
total 120
drwxr-xr-x  7 ccspapp ccsp 4096 Feb  2 14:21 base
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 boot
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 class
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 cluster
drwxr-xr-x  6 ccspapp ccsp 4096 Feb  2 14:21 codetools
drwxr-xr-x  6 ccspapp ccsp 4096 Feb  2 14:21 compiler
drwxr-xr-x  6 ccspapp ccsp 4096 Feb  2 14:21 datasets
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 foreign
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 graphics
drwxr-xr-x 11 ccspapp ccsp 4096 Feb  2 14:21 grDevices
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 grid
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 KernSmooth
drwxr-xr-x 10 ccspapp ccsp 4096 Feb  2 14:21 lattice
drwxr-xr-x 10 ccspapp ccsp 4096 Feb  2 14:21 MASS
drwxr-xr-x 12 ccspapp ccsp 4096 Feb  2 14:21 Matrix
drwxr-xr-x  7 ccspapp ccsp 4096 Feb  2 14:21 methods
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 mgcv
drwxr-xr-x 11 ccspapp ccsp 4096 Feb  2 14:21 nlme
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 nnet
drwxr-xr-x  8 ccspapp ccsp 4096 Feb  2 14:21 parallel
drwxr-xr-x 10 ccspapp ccsp 4096 Feb  2 14:21 rpart
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 spatial
drwxr-xr-x  7 ccspapp ccsp 4096 Feb  2 14:21 splines
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 stats
drwxr-xr-x  6 ccspapp ccsp 4096 Feb  2 14:21 stats4
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 survival
drwxr-xr-x  9 ccspapp ccsp 4096 Feb  2 14:21 tcltk
drwxr-xr-x  7 ccspapp ccsp 4096 Feb  2 14:21 tools
drwxr-xr-x 21 ccspapp ccsp 4096 Feb  2 14:21 translations
drwxr-xr-x 10 ccspapp ccsp 4096 Feb  2 14:21 utils

Our base Spack build will need to do the same.

balston commented 1 year ago

Trying a build using:

spack install -v r external-lapack=on rmath=on X=on %gcc@12.2.0 2>&1 | tee R-OpenBLAS-build.log
balston commented 1 year ago

It didn't like that:

==> Error: invalid values for variant "X" in package "r": ['on']

and off is not valid either!

giordano commented 1 year ago

Try with True or False? Looks like a Python boolean value: https://github.com/spack/spack/blob/42a452d54c8a25f9f415fef8cf9e3a5c64b7a46a/var/spack/repos/builtin/packages/r/package.py#L67

balston commented 1 year ago

This is working?

spack install -v r +X  +external-lapack +rmath %gcc@12.2.0 2>&1 | tee R-OpenBLAS-build.log
balston commented 1 year ago

I've now been able to check what the R binary is linked against:

ldd exec/R
        linux-vdso.so.1 =>  (0x00007ffcfb9d9000)
        libR.so => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/r-4.2.2-hwt3bbfv4dv6luxueescmc7u5ghj53wn/rlib/R/lib/libR.so (0x00007fdeb8de6000)
        libRblas.so => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/r-4.2.2-hwt3bbfv4dv6luxueescmc7u5ghj53wn/rlib/R/lib/libRblas.so (0x00007fdeb94b1000)
        libgomp.so.1 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-11.2.1/gcc-12.2.0-oklbzl53opxjovn2x3itkvs2xscuaz5m/lib64/libgomp.so.1 (0x00007fdeb9468000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fdeb8bca000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fdeb87fc000)
        libgfortran.so.5 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-11.2.1/gcc-12.2.0-oklbzl53opxjovn2x3itkvs2xscuaz5m/lib64/libgfortran.so.5 (0x00007fdeb8536000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fdeb8234000)
        libquadmath.so.0 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-11.2.1/gcc-12.2.0-oklbzl53opxjovn2x3itkvs2xscuaz5m/lib64/libquadmath.so.0 (0x00007fdeb9409000)
        libintl.so.8 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/gettext-0.21.1-avyaqb5gtb2yfk4262q6q3swa3z34tgh/lib/libintl.so.8 (0x00007fdeb93fa000)
        libreadline.so.8 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/readline-8.1.2-d7ogz4wfuefnon5cti7aqz7zbfnir5u3/lib/libreadline.so.8 (0x00007fdeb93a2000)
        libpcre2-8.so.0 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/pcre2-10.39-2g5cdagi5wef7dpf5idclsvigrww6upm/lib/libpcre2-8.so.0 (0x00007fdeb9347000)
        liblzma.so.5 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/xz-5.2.7-2b7yd37lnym4hhlw5tpw7uc5aekheq3a/lib/liblzma.so.5 (0x00007fdeb931d000)
        libbz2.so.1.0 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/bzip2-1.0.8-vvybqn5jal5kwe22slubl3g26r2zjxrz/lib/libbz2.so.1.0 (0x00007fdeb930a000)
        libz.so.1 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/zlib-1.2.13-cqggwce3maum5sf7x65daoumlwtwpfhy/lib/libz.so.1 (0x00007fdeb92ef000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fdeb802c000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fdeb7e28000)
        libiconv.so.2 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/libiconv-1.16-nqpyyn4i47ufx2amt7kfppacaohuyqyx/lib/libiconv.so.2 (0x00007fdeb7d2a000)
        libicuuc.so.67 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/icu4c-67.1-5e6wm3bxyqzp2afwbvzvkmp4yiyj4hmo/lib/libicuuc.so.67 (0x00007fdeb7adc000)
        libicui18n.so.67 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/icu4c-67.1-5e6wm3bxyqzp2afwbvzvkmp4yiyj4hmo/lib/libicui18n.so.67 (0x00007fdeb7722000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fdeb92a4000)
        libopenblas.so.0 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/openblas-0.3.21-ullkfu65x56y73aykjal3772g3w3riwy/lib/libopenblas.so.0 (0x00007fdeb6b0e000)
        libgcc_s.so.1 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-11.2.1/gcc-12.2.0-oklbzl53opxjovn2x3itkvs2xscuaz5m/lib/../lib64/libgcc_s.so.1 (0x00007fdeb92c8000)
        libncursesw.so.6 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/ncurses-6.3-w7dsx27blrvzeibrupabevnqgsh3nyqd/lib/libncursesw.so.6 (0x00007fdeb6ad4000)
        libtinfow.so.6 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/ncurses-6.3-w7dsx27blrvzeibrupabevnqgsh3nyqd/lib/libtinfow.so.6 (0x00007fdeb6a92000)
        libicudata.so.67 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/icu4c-67.1-5e6wm3bxyqzp2afwbvzvkmp4yiyj4hmo/lib/libicudata.so.67 (0x00007fdeb4f7b000)
        libstdc++.so.6 => /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gc

The OpenBLAS is the serial version created earlier so the build looks like it worked as I wanted.

Next step is to see if R actually runs!

balston commented 1 year ago

These are the current set of R recommended packages that are expected to be installed with each R installation:

MASS lattice Matrix nlme survival boot cluster codetools foreign KernSmooth rpart class nnet spatial mgcv

They should all be available in Spack with names of the form r-package so r-codetools for example but all names are lowercase. So to install them I think we just need to do:

spack install -v r-mass %gcc@12.2.0 2>&1 | tee -a R-OpenBLAS-recommended.log
balston commented 1 year ago

So running the above has installed MASS in a separate library directory:

 ls -l  /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/r-mass-7.3-58.1-hfx7etwbhri2yzccc5vap5mdxhtqppmy/rlib/R/library/MASS/
-rw-r--r-- 1 ccaabaa ccaas2   691 Feb 22 15:30 CITATION
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 data
-rw-r--r-- 1 ccaabaa ccaas2  1453 Feb 22 15:30 DESCRIPTION
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 help
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 html
-rw-r--r-- 1 ccaabaa ccaas2 10135 Feb 22 15:30 INDEX
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 libs
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 Meta
-rw-r--r-- 1 ccaabaa ccaas2  6404 Feb 22 15:30 NAMESPACE
-rw-r--r-- 1 ccaabaa ccaas2 11060 Feb 22 15:30 NEWS
drwxr-sr-x 8 ccaabaa ccaas2 33280 Feb 22 15:30 po
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 R
drwxr-sr-x 2 ccaabaa ccaas2 33280 Feb 22 15:30 scripts

I thought it would put a symlink in the main R library directory but it doesn't. This is obviously not the correct approach!

balston commented 1 year ago

So there are now 2 module files for R:

r-4.2.2-gcc-12.2.0-hwt3bbf
r-mass-7.3-58.1-gcc-12.2.0-hfx7etw

and the MASS one adds:

prepend-path    R_LIBS /lustre/scratch/ccaabaa/apps/spack-test/spack/opt/spack/linux-rhel7-cascadelake/gcc-12.2.0/r-mass-7.3-58.1-hfx7etwbhri2yzccc5vap5mdxhtqppmy/rlib/R/library

to the beginning of the R_LIBS path. If we did this for all the approx 280 additional R packages we have the R_LIBS path would become massive as would the number of modules to load in the R bundle.

This is not what we want.