installing from source does not work when using system BLAS/LAPACK

[brendanf]

There seems to be some issue installing source packages which link to BLAS/LAPACK, when R is using system versions of those packages instead of libRlapack.so and libRblas.so. In my case:

>systemInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux 8.7 (Ootpa)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblaso-r0.3.15.so;  LAPACK version 3.9.0

--etc---

I have found that such packages compile and install without problems when I install them in the same R session where I first run renv::init(), but not in any subsequent session (i.e., not in a session where the renv project is loaded on startup).

For example:

$ mkdir test
$ cd test
$ R --quiet
> renv::init()
The following package(s) will be updated in the lockfile:

# CRAN -----------------------------------------------------------------------
- renv   [* -> 1.0.2]

The version of R recorded in the lockfile will be updated:
- R      [* -> 4.3.1]

- Lockfile written to "~/projects/test/renv.lock".
- renv activated -- please restart the R session.
> renv::install("expm")
# Downloading packages -------------------------------------------------------
- Downloading expm from CRAN ...                OK [file is up to date]
Successfully downloaded 1 package in 0.49 seconds.

The following package(s) will be installed:
- expm [0.999-7]
These packages will be installed into "~/R/x86_64-redhat-linux-gnu-library/4.3".

Do you want to proceed? [Y/n]: Y
# Installing packages --------------------------------------------------------
- Installing expm ...                           OK [built from source and cached in 4.0s]
Successfully installed 1 package in 4.1 seconds.
> renv::purge("expm")
The following packages will be purged from the cache:
- expm 0.999-7 [Hash: af0c5d9f9ece61d4723904c7b2b49b4d]

Do you want to proceed? [Y/n]: Y

- Removed 1 package from the cache.
> q()
Save workspace image? [y/n/c]: n
$ R --quiet
- Project '~/projects/test' loaded. [renv 1.0.2]
> renv::install("expm")
# Downloading packages -------------------------------------------------------
- Downloading expm from CRAN ...                OK [file is up to date]
Successfully downloaded 1 package in 1 second.

The following package(s) will be installed:
- expm [0.999-7]
These packages will be installed into "~/projects/test/renv/library/R-4.3/x86_64-redhat-linux-gnu".

Do you want to proceed? [Y/n]: Y

# Installing packages --------------------------------------------------------
- Installing expm ...                           FAILED
/usr/lib64/R/bin/R --vanilla -s -f '/home/brfurnea/tmp/RtmpWLAvRk/renv-install-368c749ae66f'
================================================================================

Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/home/brfurnea/projects/test/renv/staging/1/expm/libs/expm.so':
  libRlapack.so: cannot open shared object file: No such file or directory
Calls: loadNamespace -> library.dynam -> dyn.load
Execution halted

Error: error testing if 'expm' can be loaded [error code 1]
Traceback (most recent calls last):
13: renv::install("expm")
12: renv_install_impl(records)
11: renv_install_staged(records)
10: renv_install_default(records)
 9: handler(package, renv_install_package(record))
 8: renv_install_package(record)
 7: withCallingHandlers(renv_install_package_impl(record), error = function(e) writef("FAILED"))
 6: renv_install_package_impl(record)
 5: withCallingHandlers(if (isbin) renv_install_test(package), error = function(err) unlink(installpath, 
        recursive = TRUE))
 4: renv_install_test(package)
 3: renv_system_exec(command = R(), args = c("--vanilla", "-s", "-f", 
        renv_shell_path(script)), action = sprintf("testing if '%s' can be loaded", 
        package))
 2: abort(sprintf("error %s [error code %i]", action, status), body = renv_system_exec_details(command, 
        args, output))
 1: stop(fallback)
> 

I have noticed the problem since upgrading to R 4.3 from 4.2 (via my system package manager) but it's possible that I was also using an older version of renv before.

[brendanf]

I will add that I am able to get new packages installed into existing renv projects using this as a workaround. I make a temporary "project", initialize renv, and install the package, which copies it to the cache. I can then install it in another project, where it will simply link to the working version in the cache rather than attempting to install from source.

[kevinushey]

Thanks for the bug report -- unfortunately, I wasn't able to reproduce. I was testing on an Ubuntu 22.04 VM with:

> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: aarch64-unknown-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/aarch64-linux-gnu/blas/libblas.so.3.10.0
LAPACK: /usr/lib/aarch64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.2  tools_4.1.2     renv_1.0.2.9000

And so similarly using a system-installed BLAS library. Can you confirm what output you see if you run:

options(renv.config.install.verbose = TRUE)
renv::install("expm", rebuild = TRUE)

[kevinushey]

The output of renv::diagnostics() may also be helpful.

[brendanf]

Thanks for checking it out Kevin! It's entirely possible that there is some issue with my R installation that I'm not able to figure out.

Here are the results of the commands you suggested:

> options(renv.config.install.verbose = TRUE)
> renv::install("expm", rebuild = TRUE)
# Downloading packages -------------------------------------------------------
- Downloading expm from CRAN ...                OK [file is up to date]
Successfully downloaded 1 package in 4.4 seconds.

The following package(s) will be installed:
- expm [0.999-7]
These packages will be installed into "~/projects/test/renv/library/R-4.3/x86_64-redhat-linux-gnu".

Do you want to proceed? [Y/n]: Y

# Installing packages --------------------------------------------------------
- Installing expm ...                           * installing *binary* package ‘expm’ ...
* DONE (expm)
FAILED
/usr/lib64/R/bin/R --vanilla -s -f '/home/brfurnea/tmp/RtmpK1zFw8/renv-install-3f765ed8f4a0'
================================================================================

Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/home/brfurnea/projects/test/renv/staging/1/expm/libs/expm.so':
  libRlapack.so: cannot open shared object file: No such file or directory
Calls: loadNamespace -> library.dynam -> dyn.load
Execution halted

Error: error testing if 'expm' can be loaded [error code 1]
Traceback (most recent calls last):
13: renv::install("expm", rebuild = TRUE)
12: renv_install_impl(records)
11: renv_install_staged(records)
10: renv_install_default(records)
 9: handler(package, renv_install_package(record))
 8: renv_install_package(record)
 7: withCallingHandlers(renv_install_package_impl(record), error = function(e) writef("FAILED"))
 6: renv_install_package_impl(record)
 5: withCallingHandlers(if (isbin) renv_install_test(package), error = function(err) unlink(installpath, 
        recursive = TRUE))
 4: renv_install_test(package)
 3: renv_system_exec(command = R(), args = c("--vanilla", "-s", "-f", 
        renv_shell_path(script)), action = sprintf("testing if '%s' can be loaded", 
        package))
 2: abort(sprintf("error %s [error code %i]", action, status), body = renv_system_exec_details(command, 
        args, output))
 1: stop(fallback)
> renv::diagnostics()
Diagnostics Report [renv 1.0.2]
===============================

# Session Info ---------------------------------------------------------------
R version 4.3.1 (2023-06-16)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux 8.7 (Ootpa)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblaso-r0.3.15.so;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=fi_FI.UTF-8    LC_NUMERIC=C            LC_TIME=C              
 [4] LC_COLLATE=C            LC_MONETARY=fi_FI.UTF-8 LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=fi_FI.UTF-8    LC_NAME=C               LC_ADDRESS=C           
[10] LC_TELEPHONE=C          LC_MEASUREMENT=C        LC_IDENTIFICATION=C    

time zone: Europe/Helsinki
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.1 tools_4.3.1    renv_1.0.2    

# Project --------------------------------------------------------------------
Project path: "~/projects/test"

# Status ---------------------------------------------------------------------
No issues found -- the project is in a consistent state.

# Packages -------------------------------------------------------------------
            Library Source Lockfile Source Path Dependency
KernSmooth  2.23-21   CRAN     <NA>   <NA>  [2]       <NA>
MASS         7.3-60   CRAN     <NA>   <NA>  [2]       <NA>
Matrix      1.5-4.1   CRAN     <NA>   <NA>  [2]       <NA>
boot       1.3-28.1   CRAN     <NA>   <NA>  [2]       <NA>
class        7.3-22   CRAN     <NA>   <NA>  [2]       <NA>
cluster       2.1.4   CRAN     <NA>   <NA>  [2]       <NA>
codetools    0.2-19   CRAN     <NA>   <NA>  [2]       <NA>
foreign      0.8-84   CRAN     <NA>   <NA>  [2]       <NA>
lattice      0.21-8   CRAN     <NA>   <NA>  [2]       <NA>
mgcv         1.8-42   CRAN     <NA>   <NA>  [2]       <NA>
nlme        3.1-162   CRAN     <NA>   <NA>  [2]       <NA>
nnet         7.3-19   CRAN     <NA>   <NA>  [2]       <NA>
renv          1.0.2   CRAN    1.0.2   CRAN  [1]     direct
rpart        4.1.19   CRAN     <NA>   <NA>  [2]       <NA>
spatial      7.3-16   CRAN     <NA>   <NA>  [2]       <NA>
survival      3.5-5   CRAN     <NA>   <NA>  [2]       <NA>
utils          <NA>   <NA>     <NA>   <NA>  [2]   indirect

[1]: /home/brfurnea/projects/test/renv/library/R-4.3/x86_64-redhat-linux-gnu    
[2]: /home/brfurnea/.cache/R/renv/sandbox/R-4.3/x86_64-redhat-linux-gnu/60c4e220

# ABI ------------------------------------------------------------------------
- No ABI conflicts were detected in the set of installed packages.

# User Profile ---------------------------------------------------------------
[no user profile detected]

# Settings -------------------------------------------------------------------
List of 13
 $ bioconductor.version     : NULL
 $ external.libraries       : chr(0) 
 $ ignored.packages         : chr(0) 
 $ package.dependency.fields: chr [1:3] "Imports" "Depends" "LinkingTo"
 $ ppm.enabled              : NULL
 $ ppm.ignored.urls         : chr(0) 
 $ r.version                : NULL
 $ snapshot.type            : chr "implicit"
 $ use.cache                : logi TRUE
 $ vcs.ignore.cellar        : logi TRUE
 $ vcs.ignore.library       : logi TRUE
 $ vcs.ignore.local         : logi TRUE
 $ vcs.manage.ignores       : logi TRUE

# Options --------------------------------------------------------------------
List of 9
 $ defaultPackages                     : chr [1:6] "datasets" "utils" "grDevices" "graphics" ...
 $ download.file.method                : NULL
 $ download.file.extra                 : NULL
 $ install.packages.compile.from.source: NULL
 $ pkgType                             : chr "source"
 $ repos                               : Named chr "https://packagemanager.posit.co/cran/__linux__/centos8/latest"
  ..- attr(*, "names")= chr "CRAN"
 $ renv.consent                        : logi TRUE
 $ renv.config.install.verbose         : logi TRUE
 $ renv.verbose                        : logi TRUE

# Environment Variables ------------------------------------------------------
HOME                        = /home/brfurnea
LANG                        = en_US.UTF-8
MAKE                        = make
RENV_DEFAULT_R_ENVIRON      = <NA>
RENV_DEFAULT_R_ENVIRON_USER = <NA>
RENV_DEFAULT_R_LIBS         = <NA>
RENV_DEFAULT_R_LIBS_SITE    = /usr/local/lib/R/site-library:/usr/local/lib/R/library:/usr/lib64/R/library:/usr/share/R/library
RENV_DEFAULT_R_LIBS_USER    = /home/brfurnea/R/x86_64-redhat-linux-gnu-library/4.3
RENV_DEFAULT_R_PROFILE      = <NA>
RENV_DEFAULT_R_PROFILE_USER = <NA>
RENV_PROJECT                = /home/brfurnea/projects/test
R_LIBS                      = <NA>
R_LIBS_SITE                 = /usr/local/lib/R/site-library:/usr/local/lib/R/library:/usr/lib64/R/library:/usr/share/R/library
R_LIBS_USER                 = /home/brfurnea/projects/test/renv/library/R-4.3/x86_64-redhat-linux-gnu

# PATH -----------------------------------------------------------------------
- /usr/bin
- /home/brfurnea/.local/bin
- /home/brfurnea/bin
- /usr/share/Modules/bin
- /usr/condabin
- /usr/local/bin
- /usr/local/sbin
- /usr/bin
- /usr/sbin
- /opt/puppetlabs/bin
- /var/lib/snapd/snap/bin

# Cache ----------------------------------------------------------------------
There are a total of 318 packages installed in the renv cache.
Cache path: "~/.cache/R/renv/cache/v5/R-4.3/x86_64-redhat-linux-gnu"

[kevinushey]

The only thing that stands out is that you're apparently getting a binary installation of the package?

• installing binary package ‘expm’ ...

Are you using Posit Package Manager? If so, maybe those binaries are implicitly linking to libRlapack.so -- I'll see if I can confirm if that's true. If not, do you know where that binary is being built / produced?

[kevinushey]

I found this documentation: https://docs.posit.co/resources/install-r-source/?h=lapack#optional-configure-r-to-use-a-different-blas-library

If you want to use PPM binaries on this system, I think you need to set up a symlink for the libRlapack.so and libRblas.so libraries, since installed packages from PPM will expect those libraries to exist at that location.

[brianrepko]

Noticed the same behavior on Rs built with an external LAPACK - which is the default for R 4.3. I was trying to install Matrix. We are on HPC with modules so most of our Rs are built with external BLAS and LAPACK. We also have PPM internally as well as my CRAN repo default was using the public PPM. Changed those to the cloud CRAN and added type = "source" to my install for extra measure and then it worked.

[kevinushey]

Does setting up a symlink for those libraries as described in https://docs.posit.co/resources/install-r-source/?h=lapack#optional-configure-r-to-use-a-different-blas-library help in this case?

[brianrepko]

I don't have the rights to change that - R is built via easybuild by our HPC admin team. I could create my own directory with links and add that to LD_LIBRARY_PATH but I can't add that easily to Workbench configuration.

The easier solution is have configuration options that basically never trust a binary - always build from source into the cache

[kevinushey]

The easier solution is have configuration options that basically never trust a binary - always build from source into the cache

If I understand correctly, then you probably want to set:

RENV_CONFIG_PPM_ENABLED = FALSE

This basically turns off renv's attempts to use binary repository URLs when a PPM repository appears to be in use, so that packages would always be built from sources (assuming you're using source and not binary repository URLs with your PPM instance). See https://rstudio.github.io/renv/reference/config.html#renv-config-ppm-enabled for more details.

[brianrepko]

I'll have to look through the various configurations as we have our own internal PPM as well. I think I also have to set the option to run my ~/.Rprofile as well to setup repositories for new projects that use renv - or set option("pkgType") or only do renv::init after new project setup vs using the New Project renv checkbox. All the configuration options are there - just trying to work out the best set of them. Thanks @kevinushey

[brianrepko]

but you will need to re-think those defaults with R 4.3 not having libRlapack.so by default - it is external LAPACK by default with 4.3 and up

[kevinushey]

Thanks! I've passed this feedback to the package manager team and we're going to try to re-evaluate soon.

One more option -- I wonder if setting a custom LD_LIBRARY_PATH with libRlapack.so and libRblask.so as symlinks to your desired BLAS / LAPACK libraries would work?

[skwde]

I ran into the same problem as @brianrepko. Creating symlinks to some user controlled directory does indeed work (as @kevinushey suggested) but it is tedious and ugly.

For every project one has to do something like

module load openblas
project_libs=$(pwd)/lib
mkdir -p ${project_libs:?}
export LD_LIBRARY_PATH=${project_libs:?}:${LD_LIBRARY_PATH}
ln -sf $(pkg-config --libs-only-L openblas | sed 's#^-L\(.*\)\s$#\1#g')/libopenblas.so ${project_libs:?}/libRblas.so
ln -sf ${project_libs:?}/libRblas.so ${project_libs:?}/libRlapack.so

because libRblas.so and libRlapack.so are not created when R uses external lapack.

[kevinushey]

Thanks -- unfortunately, as far as I can see, there's not much that renv can do to assist here; any requisite changes will need to happen on the Posit Package Manager side.

[maxim-h]

Thank you @brianrepko

Changed those to the cloud CRAN and added type = "source" to my install for extra measure and then it worked.

I've been trying to help a colleague, but it was a tough, because I never had that problem. Turns out that always having an explicit CRAN mirror configured saved me all this trouble.

It's unfortunate that the issue is not exactly renv, but on our system only inside renv projects the repos is set to CRAN="https://packagemanager.posit.co/cran...". Otherwise it's CRAN ="https://cran.rstudio.com/" and there aren't any problems. So perhaps that behavior is unwarranted?

It's also a bit strange that in @brendanf renv::diagnostics() the pkgType : chr "source". So why is renv installing binary package ‘expm’ ...? Even with CRAN="https://packagemanager.posit.co/cran..." doing renv::install(..., type = "source") does work correctly.

[kevinushey]

For background...

• R sets .Platform$pkgType to "binary", or a similar string, if it was built with support for binary packages.
• CRAN does not provide binary packages for Linux builds of R. For that reason, .Platform$pkgType is always "source" for an R installation on Linux.
• However, RStudio / Posit still wanted to introduce support for binary packages on Linux. This was done with Posit Package Manager by allowing users to opt-in to using special binary repository URLs.
• As far as R is concerned, it's still trying to download and install a source package; however, when R CMD INSTALL is invoked on the downloaded package, it sees that it's actually a binary and then does the right thing.

To wit, renv::install() is built with that prior understanding in mind. We could extend that to try and force source vs. binary package installations on Linux if we detect that a Posit package manager URL is being used, but we haven't done that work yet.

[maxim-h]

@kevinushey thank you for the explanation. Makes sense.

I guess it's just the renv.config.ppm.[default/enabled] being true by default that caught me by surprise originally. Not sure if it's a good or a bad thing. Guess binary packages are better for the environment at least.