search

eddelbuettel / mkl4deb

Adding the Intel MKL to a Debian / Ubuntu system via one simple script
212 stars 37 forks source link

script.sh does not affect BLAS/LAPACK library for rocker/r-ver:3.6.1 #8

Closed nathanweeks closed 4 years ago

nathanweeks commented 4 years ago

This script works great for rocker/r-ver:3.5.3, but not for rocker/r-ver:3.6.1. While it appears to execute without error, the BLAS/LAPACK library used by R is unchanged:

$ docker run -it --rm --entrypoint=/bin/bash rocker/r-ver:3.6.1
# apt update && apt install -y --no-install-recommends gnupg wget apt-transport-https
...
# wget https://raw.githubusercontent.com/eddelbuettel/mkl4deb/c82f816/script.sh
# sh script.sh
...
# Rscript -e 'sessionInfo()'
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS/LAPACK: /usr/lib/libopenblasp-r0.2.19.so
...
eddelbuettel commented 4 years ago

I am not sure it was meant for rocker/r-ver. Adjust as needed. Debugging help welcome.

nathanweeks commented 4 years ago

It appears that rocker/r-ver:3.6.1 R was configured to use OpenBLAS:

$ docker run -it --rm rocker/r-ver:3.5.3 R CMD config --all | grep -i blas
BLAS_LIBS = -lblas
$ docker run -it --rm rocker/r-ver:3.6.1 R CMD config --all | grep -i blas
BLAS_LIBS = -lopenblas

Adding a couple extra update-alternatives --install commands seems to handle this:

update-alternatives --install /usr/lib/x86_64-linux-gnu/libopenblas.so libopenblas.so-x86_64-linux-gnu  /opt/intel/mkl/lib/intel64/libmkl_rt.so 150
update-alternatives --install /usr/lib/x86_64-linux-gnu/libopenblas.so.0 libopenblas.so.0-x86_64-linux-gnu  /opt/intel/mkl/lib/intel64/libmkl_rt.so 150
> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS/LAPACK: /opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/lib/intel64_lin/libmkl_rt.so

(tested with the latest mkl, intel-mkl-64bit-2019.5-075)

Not sure if this is general-purpose enough to add to this repo's script.sh---perhaps they could be added commented-out?

Also, off-topic, but just in case this surprise (to me) isn't somewhere: since libopenblasp is multi-threaded courtesy of OpenMP, the benchmark n <- 1e3 ; X <- matrix(rnorm(n*n),n,n); system.time(svd(X)) causes R 3.6.x to spawn a number of threads equal to the number of CPUs on the host, as that's the default behavior of libgomp. On an HPC cluster where R jobs might have been allocated/constrained via cgroups to a small fraction of the CPUs on the node, this can cause severe performance degradation (e.g., our average node has 48 processor cores---and for an R job that reserves 2 processor cores, a 48-thread R process doesn't perform so well...) . In that case, it is advised to set the OMP_NUM_THREADS environment variable to the number of CPUs that have been allocated to the R job.

eddelbuettel commented 4 years ago

I think we should just close it. Maybe you can add what you discovered to the wiki, or just post a gist. This was always an unofficial script to just see how things go.

And rocker/r-ver was, as I understand my Rocker colleague, meant for stability so doing on the fly BLAS replacement from a third-party repo is a little ... strange.

But yes, update-alternatives, properly instrumented, should cover this. (Also see https://lists.debian.org/debian-science/2019/10/msg00038.html for the latest in a longish development to completely overhaul LAPACK/BLAS for Debian ...)

nathanweeks commented 4 years ago

Thanks for the link! I'll close this issue. This probably deserves more comprehensive documentation for rocker rstudio images, but that does seem outside the scope of this repo.