search

rstudio / r-builds

an opinionated environment for compiling R
GNU General Public License v3.0
101 stars 19 forks source link

Fix BLAS portability issues with Ubuntu 20.04 R builds #238

Closed glin closed 3 months ago

glin commented 3 months ago

Issue reported by @dpastoor about some Package Manager binary packages failing to load on Ubuntu 20, like RcppArmadillo.

When we switched the Ubuntu 20 builds to use OpenBLAS once again (https://github.com/rstudio/r-builds/pull/215), this inadvertently changed R to link against OpenBLAS instead of the generic libblas.so that all Ubuntu builds should be linking against. The generic libblas.so is what allows BLAS libraries to be freely swapped on Ubuntu/Debian without breaking binaries that use it.

So Package Manager binaries that use BLAS are now broken for anyone that installed R on Ubuntu 20 before May 2024. The workaround would be to manually install OpenBLAS via apt install libopenblas0-pthread.

This was also the root cause of the Cloud issue from a few months back. Switching the BLAS dependency to OpenBLAS shouldn't have actually changed what R linked to and broken anything. I just didn't realize this back then.

So currently, Ubuntu 20 R links to libopenblas.so:

$ readelf -d $(R RHOME)/lib/libR.so | grep blas
 0x0000000000000001 (NEEDED)             Shared library: [libopenblas.so.0]

This is what R should be linking to instead, from the Ubuntu 22/24 builds:

# jammy
$ readelf -d /opt/R/4.4.0/lib/R/lib/libR.so | grep blas
 0x0000000000000001 (NEEDED)             Shared library: [libblas.so.3]

The fix was to remove the development package for OpenBLAS. Apparently if both OpenBLAS and default BLAS dev headers are present, R will choose OpenBLAS first. Alternatively we can explictly specify the linker flags via --with-blas=-lblas I think, but I didn't want to change too much.

I've added a comment in the latest Ubuntu dockerfile to document this for the future.

glin commented 3 months ago

Heads up @stevenolen, this will change the Ubuntu 20 builds but it should not break anything on Cloud. This reverts the Ubuntu 20 R binary to how it was before, linking to libblas.so, which should still be present on those images.

gaborcsardi commented 3 months ago

FWIW, 22.04 has libopenblas-base, and so do the Debian Dockerfiles.

glin commented 3 months ago

@stevenolen Yes, new R versions will continue to work either with libopenblas-dev or without libopenblas-dev. Having OpenBLAS there is better since it's faster than the default BLAS, but I think you do need to keep it anyway for the existing projects that have packages linked directly to it. Removing libopenblas-dev will probably break environments created in the last few months, and also have issues with using the PPM binaries with this issue in the last few months.

So no action required, but it'd still be good to keep an eye out when this deploys.

glin commented 3 months ago

@gaborcsardi Ah yeah, there do seem to be inconsistencies with the OpenBLAS package names across images. Apparently libopenblas-base was an old package that just aliases to libopenblas0 now, which defaults to bringing in libopenblas0-pthread. Ubuntu 24 looks fine, but I can switch the package name in the Debian 12 image for future copy-paste updates.

glin commented 3 months ago

@stevenolen Rebuilding all the Ubuntu 20 binaries for production now, so they should be updated by tomorrow.