Issue linking with LAPACKE

[ataber]

I'm having trouble with undefined references when linking with OpenBLAS. My application is compiling Eigen, and I see the following errors:

eigen/Eigen/src/Core/products/TriangularSolverMatrix_BLAS.h:88: undefined reference to `dtrsm_direct'
eigen/Eigen/src/LU/PartialPivLU_LAPACKE.h:74: undefined reference to `LAPACKE_dgetrf'
eigen/Eigen/src/QR/HouseholderQR_LAPACKE.h:59: undefined reference to `LAPACKE_dgeqrf'

The linking command I'm using is

g++ -g -O2 -static -Wl,--start-group -Wl,--end-group -fopenmp -o main [object files] /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_gnu_thread.a /opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a -lgomp -lm -ldl -L/opt/OpenBLAS/lib -lopenblas -lgfortran -lpthread -fopenmp

I'm not sure how to debug what's going on. Any thoughts? Thank you!

[martin-frbg]

Make sure that nm /opt/OpenBLAS/lib/libopenblas.so |grep LAPACKE_dgetrf shows something like

0000000000b6e910 T LAPACKE_dgetrf

Did you build OpenBLAS yourself, or are you using some prebuilt package ? (One discerning factor of the LAPACK part is that it is written in fortran, so you need a fortran compiler on the build platfom to create a libopenblas that contains both BLAS and LAPACK functionality. Seeing that you have -lgfortran on your link line you probably know this already, I am only mentioning it just in case)

[ataber]

Indeed, when I run that command I get

root@cb2afd53b6a3:/app# nm /opt/OpenBLAS/lib/libopenblas.so |grep LAPACKE_dgetrf
0000000002020c10 T LAPACKE_dgetrf
0000000002020ea0 T LAPACKE_dgetrf2
0000000002020f30 T LAPACKE_dgetrf2_work
0000000002020ca0 T LAPACKE_dgetrf_work

I did compile it myself by running the following command:

  git clone -q --branch=master git://github.com/xianyi/OpenBLAS.git && \
  (cd OpenBLAS && \
  make DYNAMIC_ARCH=1 NO_AFFINITY=1 USE_OPENMP=1 && \
  make install) && \
  echo "$OPENBLAS_ROOT/lib" > /etc/ld.so.conf.d/openblas.conf && \
  ldconfig $OPENBLAS_ROOT/lib

It looks from the CMakeLists.txt that the LAPACK code is automatically compiled and included, is this true?

Thanks!

[martin-frbg]

Yes, LAPACK is automatically included by default as long as a fortran compiler is available to build it. So your libopenblas.so looks correct, but somehow the linker saw no reason to refer to it. Maybe this question would be better placed with the Eigen developers, the only thing related to third-party LAPACK I see in their documentation is that one needs to define EIGEN_USE_LAPACKE before including the Eigen headers. (https://eigen.tuxfamily.org/dox/TopicUsingBlasLapack.html)

[martin-frbg]

BTW why would you want to use both the Intel MKL and OpenBLAS in the same program ?

[ataber]

I should note that I'm statically linking the library, so it should be libopenblas.a, but that's a minor detail (I think?).

I don't understand a ton about how the various software components fit together, but from this Eigen documentation and this MKL doc I gather that MKL relies internally on some BLAS + LAPACK subroutines.

You're right I'm going to file an issue with Eigen too.

[martin-frbg]

Yeah the libopenblas.a should contain the same functions as the .so. On the other hand, MKL provides its own implementation of BLAS and LAPACK (try the same nm command on the libmkl_intel_lp64.a) so you would normally link to either of the two but not both. (Which makes it even more curious that the reference to LAPACKE_dgetrf does not get resolved, maybe the C++ compiler (g++) is just a poor stand-in for the actual linker (ld) in your case ?)

[brada4]

Maybe you post failing compilation commands firsthand? I think it is mismatch with eigen defines flipping between different files being compiled. You dont need to configure dynamic linker if you intend to include static library in your final executable module

[brada4]

Actually your G++ line for MKL is wrong. the external .a files should be between start/end group and others including openblas are plainly not considered. (as martin said - choose one) There is no issue with openblas at all. If you take out all MKL settings openblas should be fine too.

[ataber]

Great, thank you for your help. Apologies for being dense, I appreciate your patience!