'binary' package shared objects are not compatible with the docker container

[mtmorgan]

With this

root@8fc4afe31c40:/# env|grep BIOC
BIOCONDUCTOR_VERSION=3.17
BIOCONDUCTOR_USE_CONTAINER_REPOSITORY=FALSE
BIOCONDUCTOR_NAME=bioconductor_docker
BIOCONDUCTOR_DOCKER_VERSION=3.17.34

The following shows a binary package (S4Vectors) being installed, but then failing to load; the 'missing' S4Vectors.so exists, but I guess is the wrong format for this architecture -- was it built on this image?

root@0e07810d4143:/# export BIOCONDUCTOR_USE_CONTAINER_REPOSITORY=TRUE
root@0e07810d4143:/# Rscript -e "BiocManager::install('S4Vectors'); library(S4Vectors)"
'getOption("repos")' replaces Bioconductor standard repositories, see
'help("repositories", package = "BiocManager")' for details.
Replacement repositories:
    CRAN: https://packagemanager.posit.co/cran/latest
Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21)
Installing package(s) 'S4Vectors'
also installing the dependency ‘BiocGenerics’

trying URL 'https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/BiocGenerics_0.46.0_R_x86_64-pc-linux-gnu.tar.gz'
Content type 'application/gzip' length 599777 bytes (585 KB)
==================================================
downloaded 585 KB

trying URL 'https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker/src/contrib/S4Vectors_0.38.1_R_x86_64-pc-linux-gnu.tar.gz'
Content type 'application/gzip' length 2478027 bytes (2.4 MB)
==================================================
downloaded 2.4 MB

* installing *binary* package ‘BiocGenerics’ ...
* DONE (BiocGenerics)
* installing *binary* package ‘S4Vectors’ ...
* DONE (S4Vectors)
...
Error: package or namespace load failed for ‘S4Vectors’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/usr/local/lib/R/site-library/S4Vectors/libs/S4Vectors.so':
  /usr/local/lib/R/site-library/S4Vectors/libs/S4Vectors.so: cannot open shared object file: No such file or directory
Execution halted

In contrast, source installations work...

root@0e07810d4143:/# Rscript -e "BiocManager::install('S4Vectors', type = 'source', force = TRUE); library(S4Vectors)"
'getOption("repos")' replaces Bioconductor standard repositories, see
'help("repositories", package = "BiocManager")' for details.
Replacement repositories:
    CRAN: https://packagemanager.posit.co/cran/latest
Bioconductor version 3.17 (BiocManager 1.30.20), R 4.3.0 (2023-04-21)
Installing package(s) 'S4Vectors'
trying URL 'https://bioconductor.org/packages/3.17/bioc/src/contrib/S4Vectors_0.38.1.tar.gz'
Content type 'application/x-gzip' length 838821 bytes (819 KB)
==================================================
downloaded 819 KB

* installing *source* package ‘S4Vectors’ ...
** using staged installation
** libs
using C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
...
installing to /usr/local/lib/R/site-library/00LOCK-S4Vectors/00new/S4Vectors/libs
...
* DONE (S4Vectors)

The downloaded source packages are in
    ‘/tmp/RtmpSPEeF7/downloaded_packages’
Loading required package: stats4
Loading required package: BiocGenerics
...
## success!

I don't understand exactly the details but will note that this is running the (linux) container on an M1 mac...

[almahmoud]

The binaries are not compatible with arm64 which is what you would be using by default on Mac M1 unless you use the emulator and --platform=linux/amd64. That is why binaries are disabled by default for this container. Please re-open if it seems like this is a different issue.

[vjcitn]

Is there any way to intercept this event -- to avoid shipping a binary to an incompatible container environment? It seems like it would be something BiocManager would have to check.

[mtmorgan]

Generally I was wondering whether BiocManager is able to distinguish binary architectures; I think it always points to the same repository

> BiocManager::containerRepository()
                                                                 BioCcontainers
"https://bioconductor.org/packages/3.17/container-binaries/bioconductor_docker"

Will that have to change?

[almahmoud]

Currently the binaries are off by default for the arm64 containers so this will only happen when a user knows what env variable to change and manually sets it to use them as Martin did, so I am hoping it's not a common issue except for "power users". The binaries URL does not currently support architecture, but I have a commit to BiocManager that changes that, which I'll PR after the binaries are shipped/relocated in the bucket. I am hoping that 3.18+ will have container binaries for both architectures.