search

ornladios / ADIOS2

Next generation of ADIOS developed in the Exascale Computing Program
https://adios2.readthedocs.io/en/latest/index.html
Apache License 2.0
272 stars 126 forks source link

Cannot link ADIOS2 through CMake #805

Closed williamfgc closed 5 years ago

williamfgc commented 6 years ago

@chuckatkins I have a cmake project that is trying to link with ADIOS2 after a normal make install

This is a Fortran90 project for which I have in CMakeLists.txt:

find_package(ADIOS2 REQUIRED)
...
target_link_libraries(${programfile}.x adios2::adios2_Fortran ${BLAS_LIBRARIES} MPI::MPI_Fortran )

But I get the following error when running : cmake -DADIOS2_ROOT=/opt/adios2/ ..

CMake Error at CMakeLists.txt:22 (find_package):
  By not providing "FindADIOS2.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "ADIOS2", but
  CMake did not find one.

  Could not find a package configuration file provided by "ADIOS2" with any
  of the following names:

    ADIOS2Config.cmake
    adios2-config.cmake

  Add the installation prefix of "ADIOS2" to CMAKE_PREFIX_PATH or set
  "ADIOS2_DIR" to a directory containing one of the above files.  If "ADIOS2"
  provides a separate development package or SDK, be sure it has been
  installed.

-- Configuring incomplete, errors occurred!

The ADIOS2Config.cmake file exists under adios2/lib/cmake

williamfgc commented 6 years ago

Also, is it ADIOS2_DIR or ADIOS2_ROOT? Either one will give the same error.

williamfgc commented 6 years ago

I am able to reproduce with the ADIOS2-Examples repo

williamfgc commented 6 years ago

@fbudin69500 we typically use _ROOT for the top installation directory, not when Config.cmake is under _ROOT/lib/. Also, please try with all the supported dependencies, example:

Features:
    Library Type: shared
    Build Type:   Debug
    Testing: ON
    Build Options:
      BZip2    : ON
      ZFP      : ON
      SZ       : ON
      MPI      : ON
      DataMan  : ON
      SST      : ON
      ZeroMQ   : ON
      HDF5     : ON
      ADIOS1   : ON
      Python   : ON
      Fortran  : ON
      SysVShMem: ON

-- Configuring done
-- Generating done
-- Build files have been written to: /home/wgodoy/workspace/build
fbudin69500 commented 6 years ago

Beside the fact that ADIOS2-Examples, or more specifically ADIOS2-Examples/cpp/heat did not want to build correctly, I was able to configure the project on my machine with CMake 3.10.2 when using ADIOS2_DIR. To be able to configure the example project using ADIOS2_ROOT, but only with the latest version of CMake 3.12.2. It may work with other versions that are a little bit older, but fails with CMake 3.10.2. A problem has been corrected recently in CMake concerning the _ROOT variable. NOTE: I did not have all the modules activated as some were causing problems on my machine (ADIOS1, SZ, Python), but I don't think it matters for this specific issue.

williamfgc commented 6 years ago

@fbudin69500 are all optional dependencies turned on? Can you share the code so we can follow?. The promise is ADIOS2 will work with cmake v3.6. None of the supercomputers at DOE has a recent cmake version.

williamfgc commented 6 years ago

It matters, in fact it's the most important feature to link with our dependencies. Being able to link any version of adios2 (with any dependency) needed by the user for its cmake project.

chuckatkins commented 6 years ago

PackageName_ROOT and PackageName_DIR have different meanings. _ROOT is builtin to CMake 3.12 for find modules while < 3.12 used custom logic in find modules to use it. _DIR, on the other hand, has been built into CMake for a very long time and is used to directly consume a PackageNameConfig.cmake file. There are different expectations if its used as a CMake variable vs env variable. As a CMake variable via -DPackageName_DIR needs to point to the directory containing PackageNameConfig.cmake, while as an env var PackageName_DIR, it needs to point to the installation prefix. I'm abe to configure the ADIOS2-Examples repo via:

# Set the env var to the ADIOS2 install prefix.
$ export ADIOS2_DIR=~/Code/adios2/install/work
$ cmake ../../source
cmake ../../source/master
-- The C compiler identification is GNU 8.1.1
-- The CXX compiler identification is GNU 8.1.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /usr/lib64/openmpi/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /usr/lib64/openmpi/lib/libmpi_cxx.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- Found ADIOS2: 2.2.0  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/chuck/Code/adios2/examples/build/master
$
pnorbert commented 6 years ago

How does this work for static builds? I can reproduce your successful configuration with a shared build on my VM, but not on Titan with a static build. ADIOS is found, but not its dependencies. I am using the latest version of cmake on Titan, 3.9.0. First, it does not find HDF5:

pnorbert@titan-ext6:/ccs/proj/e2e/pnorbert/ADIOS/ADIOS2-Examples/build $ ADIOS2_DIR=/ccs/proj/e2e/pnorbert/ADIOS/ADIOS2/build.titan.gnu/install cmake ..
-- The C compiler identification is GNU 6.3.0
-- The CXX compiler identification is GNU 6.3.0
-- Cray Programming Environment 2.5.13 C
-- Check for working C compiler: /opt/cray/craype/2.5.13/bin/cc
-- Check for working C compiler: /opt/cray/craype/2.5.13/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Cray Programming Environment 2.5.13 CXX
-- Check for working CXX compiler: /opt/cray/craype/2.5.13/bin/CC
-- Check for working CXX compiler: /opt/cray/craype/2.5.13/bin/CC -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /opt/cray/craype/2.5.13/bin/cc  
-- Found MPI_CXX: /opt/cray/craype/2.5.13/bin/CC  
-- Found BZip2: /usr/lib64/libbz2.a (found version "1.0.5") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
CMake Error at /autofs/nccs-svm1_sw/titan/.swci/0-login/opt/spack/20170612/linux-suse_linux11-x86_64/gcc-4.3.4/cmake-3.9.0-owxiriblogovogl5zbrg45ulm3ln34cx/share/cmake-3.9/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find HDF5 (missing: HDF5_LIBRARIES HDF5_INCLUDE_DIRS C)
Call Stack (most recent call first):
  /autofs/nccs-svm1_sw/titan/.swci/0-login/opt/spack/20170612/linux-suse_linux11-x86_64/gcc-4.3.4/cmake-3.9.0-owxiriblogovogl5zbrg45ulm3ln34cx/share/cmake-3.9/Modules/FindPackageHandleStandardArgs.cmake:377 (_FPHSA_FAILURE_MESSAGE)
  /autofs/nccs-svm1_sw/titan/.swci/0-login/opt/spack/20170612/linux-suse_linux11-x86_64/gcc-4.3.4/cmake-3.9.0-owxiriblogovogl5zbrg45ulm3ln34cx/share/cmake-3.9/Modules/FindHDF5.cmake:839 (find_package_handle_standard_args)
  /autofs/nccs-svm1_sw/titan/.swci/0-login/opt/spack/20170612/linux-suse_linux11-x86_64/gcc-4.3.4/cmake-3.9.0-owxiriblogovogl5zbrg45ulm3ln34cx/share/cmake-3.9/Modules/CMakeFindDependencyMacro.cmake:37 (find_package)
  /ccs/proj/e2e/pnorbert/ADIOS/ADIOS2/build.titan.gnu/install/lib/cmake/adios2/ADIOS2ConfigCommon.cmake:35 (find_dependency)
  /ccs/proj/e2e/pnorbert/ADIOS/ADIOS2/build.titan.gnu/install/lib/cmake/adios2/ADIOS2Config.cmake:22 (include)
  cpp/heat/CMakeLists.txt:26 (find_package)

If I have HDF5_DIR set (module load cray-hdf5-parallel), it finds HDF5, but not EVPath:

$ cmake ..
-- Cray Programming Environment 2.5.13 C
-- Cray Programming Environment 2.5.13 CXX
-- HDF5: Using hdf5 compiler wrapper for all C compiling
-- Found HDF5: Included by compiler wrappers (found version ".1") found components:  C 
CMake Error at /autofs/nccs-svm1_sw/titan/.swci/0-login/opt/spack/20170612/linux-suse_linux11-x86_64/gcc-4.3.4/cmake-3.9.0-owxiriblogovogl5zbrg45ulm3ln34cx/share/cmake-3.9/Modules/CMakeFindDependencyMacro.cmake:37 (find_package):
  By not providing "FindEVPath.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "EVPath", but
  CMake did not find one.

  Could not find a package configuration file provided by "EVPath" with any
  of the following names:

    EVPathConfig.cmake
    evpath-config.cmake

Note that EVPath is built as part of ADIOS2, so it should find it from ADIOS2_DIR. Also Note that EVPathConfig.cmake is located in ADIOS2_DIR/lib/cmake/adios2/deps.

As a user, I want to point to the ADIOS installation directory and let cmake figure out the dependencies.

pnorbert commented 6 years ago

Hoping to move on with my task to do, I built a shared lib of adios without mpi and any other dependencies. Unfortunately, the installed ADIOS2Config.cmake is invalid and thus I cannot use this shared install either. 

$ cmake -DCMAKE_INSTALL_PREFIX=$PWD/install -DBUILD_SHARED_LIBS=ON  -DCMAKE_BUILD_TYPE=RelWithDebInfo -DADIOS2_USE_MPI=OFF -DADIOS2_USE_SST=OFF -DADIOS2_USE_Fortran=OFF -DADIOS2_USE_Python=OFF -DADIOS2_USE_ADIOS1=OFF -DADIOS2_USE_HDF5=OFF -DADIOS2_BUILD_TESTING=OFF -DADIOS2_BUILD_EXAMPLES_EXPERIMENTAL=OFF -DADIOS2_USE_ZeroMQ=OFF -DADIOS2_USE_BZip2=OFF -DADIOS2_USE_ZFP=OFF -DADIOS2_USE_SZ=OFF   ..
$ cat install/lib/cmake/adios2/ADIOS2Config.cmake
list(INSERT CMAKE_MODULE_PATH 0 "${CMAKE_CURRENT_LIST_DIR}/Modules")

if( AND NOT )
  if(NOT )
    set(atl_DIR lib/cmake/adios2/deps)
  endif()

  if(NOT )
    if(NOT )
      set(dill_DIR lib/cmake/adios2/deps)
    endif()
    set(ffs_DIR lib/cmake/adios2/deps)
  endif()

  if(NOT )
    set(enet_DIR lib/cmake/adios2/deps)
  endif()

  set(EVPath_DIR lib/cmake/adios2/deps)
endif()

include("${CMAKE_CURRENT_LIST_DIR}/ADIOS2ConfigCommon.cmake")
CMake Error at /ccs/proj/e2e/pnorbert/ADIOS/ADIOS2/build.titan.gnu-nompi-shared/install/lib/cmake/adios2/ADIOS2Config.cmake:3 (if):
  if given arguments:

    "AND" "NOT"

  Unknown arguments specified
chuckatkins commented 5 years ago

Fixed by #1081