Error in linking dynamic library when compiling ABACUS with DeePMD-kit C++ interface

YuLiu98 commented 2 years ago

Summary

Note that I succeed in compiling ABACUS with DeePMD-kit C++ interface through Makefiles, and the results are the same as those of Lammps.

However, when I try to compile ABACUS with DeePMD-kit C++ interface through Cmake, there are errors in linking the dynamic library.

Maybe DeePMD-kit could provide a cmake config for the importation of DeePMD-kit into other packages such as ABACUS.

DeePMD-kit Version

DeePMD-kit v2.1.5

TensorFlow Version

TensorFlow v2.9.0

Python Version, CUDA Version, GCC Version, LAMMPS Version, etc

Python 3.10.4

CUDA 11.6

GCC 9.4.0

Details

1) I succeed in compiling ABACUS with DeePMD-kit C++ interface through Makefiles according to the tutorial: https://github.com/deepmodeling/deepmd-kit/blob/master/doc/inference/cxx.md.

2) Now ABACUS in this branch can compile with DeePMD-kit: https://github.com/YuLiu98/abacus-develop/tree/esolver. By specifying DeeP_DIR and TensorFlow_DIR in source/Makefile.vars, ABACUS is combined with DeePMD-kit through the inference.

3) I compile ABACUS by compiling with cmake by adding a part in CmakeLists.txt to find DeePMD-kit and TensorFlow: CMakeLists.txt

The part is as follows:

if(DEFINED DeePMD_DIR)
  add_compile_definitions(
    __DPMD 
    HIGH_PREC
  )
  add_compile_options(-Wl,--no-as-needed)
  find_library(deepmd_cc
    NAMES deepmd_cc
    PATHS ${DeePMD_DIR}/lib
  )
  find_library(DeePMD_OP
    NAMES deepmd_op
    PATHS ${DeePMD_DIR}/lib
  )
  find_library(deepmd_op_cuda
    NAMES deepmd_op_cuda
    PATHS ${DeePMD_DIR}/lib
  )
  find_library(tensorflow_cc
    NAMES tensorflow_cc
    PATHS ${DeePMD_DIR}/lib
  )
  include_directories(${DeePMD_DIR}/include)
  target_link_libraries(${ABACUS_BIN_NAME} 
    ${deepmd_cc}
    ${DeePMD_OP}
    ${deepmd_op_cuda}
    ${tensorflow_cc}
  )
  if(NOT DEFINED TensorFlow_DIR)
    message(FATAL_ERROR "Please define TensorFlow_DIR!")
  endif()
endif()

if(DEFINED TensorFlow_DIR)
  include_directories(${TensorFlow_DIR}/include)
  if(NOT DEFINED DeePMD_DIR)
    message(FATAL_ERROR "Please define DeePMD_DIR!")
  endif()
endif()

This is the cmake.log, which contains the error information: cmake.log

njzjz commented 2 years ago

The error is probably caused by inconsitent versions of MKL. Did you enable MKL for TensorFlow?

CMake Warning at CMakeLists.txt:30 (add_executable):
  Cannot generate a safe runtime search path for target abacus because files
  in some directories may conflict with libraries in implicit directories:

    runtime library [libmpicxx.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmpifort.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmpi.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib/release may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libiomp5.so] in /opt/intel/oneapi/compiler/2021.3.0/linux/compiler/lib/intel64_lin may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_intel_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_intel_thread.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_core.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_scalapack_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_blacs_intelmpi_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib

  Some of these libraries may not be found correctly.

YuLiu98 commented 2 years ago

The error is probably caused by inconsitent versions of MKL. Did you enable MKL for TensorFlow?

CMake Warning at CMakeLists.txt:30 (add_executable):
  Cannot generate a safe runtime search path for target abacus because files
  in some directories may conflict with libraries in implicit directories:

    runtime library [libmpicxx.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmpifort.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmpi.so.12] in /opt/intel/oneapi/mpi/2021.3.0/lib/release may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libiomp5.so] in /opt/intel/oneapi/compiler/2021.3.0/linux/compiler/lib/intel64_lin may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_intel_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_intel_thread.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_core.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_scalapack_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib
    runtime library [libmkl_blacs_intelmpi_lp64.so.1] in /opt/intel/oneapi/mkl/2021.3.0/lib/intel64 may be hidden by files in:
      /home/liuyu/deepmd-kit/lib

  Some of these libraries may not be found correctly.

I could compile ABACUS with DeePMD-kit through Makefile, so I think MKL is enabled for TensorFlow.

njzjz commented 2 years ago

From your log I see multiple MKL files:

/home/liuyu/deepmd-kit/lib/libmkl_sequential.so.1
/opt/intel/oneapi/mkl/2021.3.0/lib/intel64/libmkl_intel_lp64.so

Maybe they have different versions and thus are conflict.

YuLiu98 commented 2 years ago

From your log I see multiple MKL files:
/home/liuyu/deepmd-kit/lib/libmkl_sequential.so.1
/opt/intel/oneapi/mkl/2021.3.0/lib/intel64/libmkl_intel_lp64.so
Maybe they have different versions and thus are conflict.

OK, thank you, I succeeded in compiling it on another machine.

njzjz commented 1 year ago

A cmake config is added in #2049.

deepmodeling / deepmd-kit