search

szcf-weiya / techNotes

My notes about tech (Linux/R/Julia/Python/Cpp/etc.)
https://tech.hohoweiya.xyz/
11 stars 7 forks source link

chpc R/4.1: fail to install glmnet #47

Closed szcf-weiya closed 2 years ago

szcf-weiya commented 2 years ago 
@chpc-sandbox ~]$ module load R/4.1 
Loading advisor version 2022.1.0
Loading ccl version 2021.6.0
Loading clck version 2021.6.0
Loading compiler version 2022.1.0
Loading tbb version 2021.6.0
Loading compiler-rt version 2022.1.0
Loading oclfpga version 2022.1.0
  Load "debugger" to debug DPC++ applications with the gdb-oneapi debugger.
  Load "dpl" for additional DPC++ APIs: https://github.com/oneapi-src/oneDPL
Loading dal version 2021.6.0
Loading debugger version 2021.6.0
Loading dev-utilities version 2021.6.0
Loading dnnl version 2022.1.0
Loading dpct version 2022.1.0
Loading dpl version 2021.7.0
Loading icc version 2022.1.0
Loading inspector version 2022.1.0
Loading itac version 2021.6.0
Loading mkl version 2022.1.0
Loading mpi version 2021.6.0
Loading vpl version 2022.1.0

Loading R/4.1
  Loading requirement: pcre2/10.31 gcc/10.3.0 advisor/latest ccl/latest clck/latest tbb/latest compiler-rt/latest oclfpga/latest compiler/latest dal/latest debugger/latest
    dev-utilities/latest dnnl/latest dpct/latest dpl/latest icc/latest inspector/latest itac/latest mkl/latest mpi/latest vpl/latest intel/2022.2

I have installed glmnet before, but it failed to library it,

> library(glmnet)
Loading required package: Matrix
Error: package or namespace load failed for ‘glmnet’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/storage01/users/*********/R/x86_64-pc-linux-gnu-library/4.1/glmnet/libs/glmnet.so':
  /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /storage01/users/***********/R/x86_64-pc-linux-gnu-library/4.1/glmnet/libs/glmnet.so)

Then try to re-install it, but it failed,

> install.packages("glmnet")
Error: C++14 standard requested but CXX14 is not defined
* removing ‘/storage01/users/***********/R/x86_64-pc-linux-gnu-library/4.1/glmnet’
* restoring previous ‘/storage01/users/***********/R/x86_64-pc-linux-gnu-library/4.1/glmnet’

The downloaded source packages are in
    ‘/tmp/RtmpVqAfBX/downloaded_packages’
Warning message:
In install.packages("glmnet") :
  installation of package ‘glmnet’ had non-zero exit status

Try to add ~/.R/Makevar as suggested in https://github.com/stan-dev/rstan/issues/892, it can start to compile but throw a new problem

g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c RcppExports.cpp -o RcppExports.o
g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c elnet_exp.cpp -o elnet_exp.o
gfortran  -fpic  -O3 -march=native -c glmnet5dpclean.f -o glmnet5dpclean.o
g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c glmnet_init.cpp -o glmnet_init.o
g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c internal.cpp -o internal.o
gcc -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include   -fpic  -O3 -march=native -c pb.c -o pb.o
g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c pb_exp.cpp -o pb_exp.o
gfortran  -fpic  -O3 -march=native -c wls.f -o wls.o
g++ -std=c++14 -I"/opt/share/R/4.1.2/centos/intel/lib64/R/include" -DNDEBUG  -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/RcppEigen/include' -I'/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I/usr/local/include  -Iglmnetpp/include -Iglmnetpp/src -Iglmnetpp/test  -fPIC  -O3 -march=native -Wno-ignored-attributes -c wls_exp.cpp -o wls_exp.o
g++ -std=c++14 -shared -L/usr/local/lib64 -o glmnet.so RcppExports.o elnet_exp.o glmnet5dpclean.o glmnet_init.o internal.o pb.o pb_exp.o wls.o wls_exp.o -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -lirc_s -ldl
/usr/bin/ld: cannot find -lifport
/usr/bin/ld: cannot find -lifcoremt
/usr/bin/ld: cannot find -limf
/usr/bin/ld: cannot find -lsvml
/usr/bin/ld: cannot find -lipgo
/usr/bin/ld: cannot find -lirc
collect2: error: ld returned 1 exit status
make: *** [glmnet.so] Error 1
ERROR: compilation failed for package ‘glmnet’
* removing ‘/storage01/users/*************/R/x86_64-pc-linux-gnu-library/4.1/glmnet’
* restoring previous ‘/storage01/users/***********/R/x86_64-pc-linux-gnu-library/4.1/glmnet’

The downloaded source packages are in
    ‘/tmp/RtmpuEEnMh/downloaded_packages’
Warning message:
In install.packages("glmnet") :
  installation of package ‘glmnet’ had non-zero exit status

A guess

Note that module load R/4.1 would load intel, 

$ pwd
/opt/share/module-environment/5.0.1/modulefiles/R
[s**********@chpc-sandbox R]$ cat 4.1 
........................

module load pcre2
module load intel

.......................

but intel does not specify a version, however, there has been a newly added intel version recently (on Sep. 16), so it would load the new version 2022.2, which might not be compatible with the current R version.

[s**********@chpc-sandbox intel]$ pwd
/opt/share/module-environment/5.0.1/modulefiles/intel
[s**********@chpc-sandbox intel]$ ll
total 48
-rw-r--r--. 1 root root 5441 Aug 20  2021 2018u2
-rw-r--r--. 1 root root 5183 Aug 20  2021 2019u3
-rw-r--r--. 1 root root 5215 Aug 20  2021 2019u4
-rw-r--r--. 1 root root 5699 Aug 20  2021 2020u1
-rw-r--r--. 1 root root 7607 Aug 20  2021 2020u4
-rw-r--r--. 1 root root 1311 Feb  8  2022 2021.4
-rw-r--r--. 1 root root 2461 Sep 16 13:05 2022.2

If so, a direct solution would be specify the version of intel in the modulefile of R/4.1

szcf-weiya commented 2 years ago
  1. copy R/4.1 to the local module file folder, and replace intel with intel/2021.4
  2. to fulfill C++14, use gcc>6.1 (https://gcc.gnu.org/projects/cxx-status.html), so module load gcc/10.3.0, but it throws 
    ** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘glmnet’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-glmnet/00new/glmnet/libs/glmnet.so':
/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-glmnet/00new/glmnet/libs/glmnet.so: undefined symbol: _gfortran_runtime_error_at
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/glmnet’
* restoring previous ‘/storage01/users/s**********/R/x86_64-pc-linux-gnu-library/4.1/glmnet’
  3. add FLIBS=-lgfortran -lm although lgfortran can be found in LD_LIBRARY_PATH
szcf-weiya commented 1 year ago

on chpc-gpu019, just use intel/2020u4 as suggested by Edward (check the email on Oct. 17), and re-install, since gcc has fulfilled the requirement

$ gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.