RELEASE_3_17: *.so shared objects from container_packages in /usr/local/lib/R/site-library packages not loading

[lima1]

Describe the bug

Not a docker expert, so apologies if I did something dumb, but when I tried updating my Dockerfile to latest RELEASE_3_17, couple of dependencies did not load. Reinstalling them with install.packages fixed it. Is it possible that there is an R conflict between the repository BiocManager uses by default?

To Reproduce

Steps to reproduce the behavior:

1. docker run -it --user rstudio bioconductor/bioconductor_docker:RELEASE_3_17 R

2. BiocManager::install("matrixStats", force=TRUE) 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for details. Replacement repositories: CRAN: https://packagemanager.posit.co/cran/latest Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21) Installing package(s) 'matrixStats' trying URL 'https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/matrixStats_0.63.0_R_x86_64-pc-linux-gnu.tar.gz' Content type 'application/x-tar' length 672206 bytes (656 KB)

   downloaded 656 KB

• installing binary package ‘matrixStats’ ...
• DONE (matrixStats)

The downloaded source packages are in ‘/tmp/RtmpAyOlQO/downloaded_packages’ Old packages: 'xml2' Update all/some/none? [a/s/n]: n > library(matrixStats) Error: package or namespace load failed for ‘matrixStats’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so': /usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so: cannot open shared object file: No such file or directory

3. > install.packages("matrixStats") ...
4. > library(matrixStats)

Works.

[vjcitn]

I couldn't reproduce this.

What is value of Sys.getenv("BIOCONDUCTOR_DOCKER_VERSION")

On Sun, Apr 30, 2023 at 2:53 PM Markus Riester @.***> wrote:

Describe the bug

Not a docker expert, so apologies if I did something dumb, but when I tried updating my Dockerfile to latest RELEASE_3_17, couple of dependencies did not load. Reinstalling them with install.packages fixed it. Is it possible that there is an R conflict between the repository BiocManager uses by default?

To Reproduce

Steps to reproduce the behavior:

1. docker run -it --user rstudio bioconductor/bioconductor_docker:RELEASE_3_17 R

2. BiocManager::install("matrixStats", force=TRUE) 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for details. Replacement repositories: CRAN: https://packagemanager.posit.co/cran/latest Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21) Installing package(s) 'matrixStats' trying URL ' https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/matrixStats_0.63.0_R_x86_64-pc-linux-gnu.tar.gz ' Content type 'application/x-tar' length 672206 bytes (656 KB)

   downloaded 656 KB

   • installing binary package ‘matrixStats’ ...
   • DONE (matrixStats)

The downloaded source packages are in ‘/tmp/RtmpAyOlQO/downloaded_packages’ Old packages: 'xml2' Update all/some/none? [a/s/n]: n

library(matrixStats) Error: package or namespace load failed for ‘matrixStats’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so': /usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so: cannot open shared object file: No such file or directory

3. install.packages("matrixStats") ...

4. library(matrixStats)

Works.

— Reply to this email directly, view it on GitHub https://github.com/Bioconductor/bioconductor_docker/issues/78, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDI5QTN7BILBIPJ5426QETXD2YJBANCNFSM6AAAAAAXRC7XSE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance  HelpLine at http://www.partners.org/complianceline http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

[lima1]

Sys.getenv("BIOCONDUCTOR_DOCKER_VERSION") [1] "3.17.33" BiocManager::install("matrixStats", force=TRUE) 'getOption("repos")' replaces Bioconductor standard repositories, see 'help("repositories", package = "BiocManager")' for details. Replacement repositories: CRAN: https://packagemanager.posit.co/cran/latest Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21) Installing package(s) 'matrixStats' trying URL 'https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/matrixStats_0.63.0_R_x86_64-pc-linux-gnu.tar.gz' Content type 'application/x-tar' length 672206 bytes (656 KB)

downloaded 656 KB

• installing binary package ‘matrixStats’ ...
• DONE (matrixStats)

The downloaded source packages are in ‘/tmp/RtmpRrOeyS/downloaded_packages’ Old packages: 'xml2' Update all/some/none? [a/s/n]: n

library(matrixStats) Error: package or namespace load failed for ‘matrixStats’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so': /usr/local/lib/R/site-library/matrixStats/libs/matrixStats.so: cannot open shared object file: No such file or directory

[vjcitn]

I do see a little different information on my attempt:

> BiocManager::install("matrixStats")
'getOption("repos")' replaces Bioconductor standard repositories, see
'help("repositories", package = "BiocManager")' for details.
Replacement repositories:
    CRAN: https://packagemanager.posit.co/cran/__linux__/jammy/latest
Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21)
Installing package(s) 'matrixStats'
trying URL 'https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/matrixStats_0.63.0_R_x86_64-pc-linux-gnu.tar.gz'
Content type 'application/x-tar' length 672206 bytes (656 KB)
==================================================
downloaded 656 KB

* installing *binary* package ‘matrixStats’ ...
* DONE (matrixStats)

The downloaded source packages are in
    ‘/tmp/RtmpnLQxLJ/downloaded_packages’
Old packages: 'xml2'
Update all/some/none? [a/s/n]: n
> library(matrixStats)

That library() call succeeded. But the "Replacement repositories:" text is different. I don't know if this matters, it wouldn't seem so, you are getting the same number of bytes in your download. Is there any chance you have a shortage of disk space so that the .so doesn't actually get placed where it belongs? We may have to wait until the container expert @almahmoud comes on line.

[almahmoud]

What type of machine do you have? Are you by any chance on an arm64 machine (eg Mac M1/M2 or linux arm)? That is my best guess for why you are experiencing the error with the binaries. If so, please try setting env BIOCONDUCTOR_USE_CONTAINER_REPOSITORY=FALSE. I will also set it by default for the arm64 image, until we add support for arm binaries, which might be next release. If not, we will have to investigate further. Could you provide the log for the source installation?

[vjcitn]

I think @almahmoud is right. I just tried to reproduce on the M1 mac and got Markus' error. sessionInfo result would have made this plain. Setting the env var solves the problem by installing only from source.

[vjcitn]

Maybe there should be an env var indicating the chip type for container?

[lima1]

Thanks both, indeed a Mac.

[almahmoud]

Maybe there should be an env var indicating the chip type for container?

You can now see it under PLATFORM env var.

[almahmoud]

https://github.com/Bioconductor/bioconductor_docker/pull/79 also sets the default for arm64 containers to not use the binary repositories. After the rebuild in ~1 hour, this should not be an issue anymore in the container with PATCH number 34.