unittestX fails to build with undefined reference

[opoplawski]

Building current git master:

[ 87%] Linking CXX executable unittestX
/usr/bin/cmake3 -E cmake_link_script CMakeFiles/unittestX.dir/link.txt --verbose=1
/opt/intel-2017/openmpi-2.1.1/bin/mpicxx   -Wall -O3 -g -pipe -Wall -Werror=format-security -fexceptions --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -axSSE3,SSSE3,SSE4.1,SSE4.2,CORE-AVX2,COMMON-AVX512  -std=c++11 -O3 -DNDEBUG -restrict -O3 -DNDEBUG   -L/opt/intel-2017/openmpi-2.1/netcdf-4.4.1/lib64 CMakeFiles/unittestX.dir/unit_tests.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTest1ElemCoordCheck.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestABLWallFunction.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestBasicKokkos.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestElemSuppAlg.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestElementDescription.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestFieldUtils.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestGetDofStatus.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestHOMasterElements.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestHex27FaceNodeOrdering.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestHexElementPromotion.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestHexMasterElements.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestHexSCVDeterminant.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestKokkosME.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestKokkosViews.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestMasterElements.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestMetricTensor.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestMovingAverage.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestNGPMasterElements.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestQuadElementPromotion.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestRealm.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSideIsInElement.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSideNodeOrdinals.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSidePCoords.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSimdBasic.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSimdContAdvElem.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSimdMomentum.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSingleHexPromotion.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestSuppAlgDataSharing.C.o CMakeFiles/unittestX.dir/unit_tests/UnitTestUtils.C.o CMakeFiles/unittestX.dir/unit_tests/algorithms/UnitTestAlgorithm.C.o CMakeFiles/unittestX.dir/unit_tests/algorithms/UnitTestLESAlgorithms.C.o CMakeFiles/unittestX.dir/unit_tests/algorithms/UnitTestMomentumBoussinesqSrcNodeSuppAlg.C.o CMakeFiles/unittestX.dir/unit_tests/algorithms/UnitTestRodiAlgorithm.C.o CMakeFiles/unittestX.dir/unit_tests/algorithms/UnitTestSSTAlgorithms.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestContinuityAdvElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestContinuityMassElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestKernelUtils.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumActuatorSrcElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumAdvDiffElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumBuoyancyBoussinesqSrcElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumBuoyancySrcElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumCoriolisSrcElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumHybridTurbElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumMassElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumNSOElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestMomentumNSOKeElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestScalarAdvDiffElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestScalarDiffElem.C.o CMakeFiles/unittestX.dir/unit_tests/kernels/UnitTestSteadyThermal3dContactSrcElem.C.o  -o unittestX -rdynamic libnalu.a /data/cora1/orion/nalu_build/install/trilinos/lib/libmuelu-adapters.a /data/cora1/orion/nalu_build/install/trilinos/lib/libmuelu-interface.a /data/cora1/orion/nalu_build/install/trilinos/lib/libmuelu.a /data/cora1/orion/nalu_build/install/trilinos/lib/libifpack2.a /data/cora1/orion/nalu_build/install/trilinos/lib/libamesos2.a -lsuperlu /data/cora1/orion/nalu_build/install/trilinos/lib/libbelostpetra.a /data/cora1/orion/nalu_build/install/trilinos/lib/libbelos.a /data/cora1/orion/nalu_build/install/trilinos/lib/libzoltan2.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtrilinosss.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_mesh_fixtures.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_tools_lib.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_transfer_impl.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_search.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_unit_main.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_unit_test_utils.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_io_util.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_io.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_mesh_base.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_topology.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_use_cases.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_registry.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_diag.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_env.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_parallel.a /data/cora1/orion/nalu_build/install/trilinos/lib/libstk_util_util.a /data/cora1/orion/nalu_build/install/lib/libboost_program_options.so /data/cora1/orion/nalu_build/install/lib/libboost_system.so /data/cora1/orion/nalu_build/install/trilinos/lib/libio_info_lib.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIonit.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIotr.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIohb.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIogn.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIovs.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIoexo_fac.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIofx.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIoex.a /data/cora1/orion/nalu_build/install/trilinos/lib/libIoss.a /data/cora1/orion/nalu_build/install/trilinos/lib/libzoltan.a /data/cora1/orion/nalu_build/install/trilinos/lib/libexodus.a -lnetcdf /data/cora1/orion/nalu_build/install/trilinos/lib/libtpi.a /data/cora1/orion/nalu_build/install/trilinos/lib/libgtest.a /data/cora1/orion/nalu_build/install/trilinos/lib/libintrepid.a /data/cora1/orion/nalu_build/install/trilinos/lib/libshards.a /data/cora1/orion/nalu_build/install/trilinos/lib/libsacado.a /data/cora1/orion/nalu_build/install/trilinos/lib/libsuplib_cpp.a /data/cora1/orion/nalu_build/install/trilinos/lib/libsuplib_c.a /data/cora1/orion/nalu_build/install/trilinos/lib/libchaco.a -lm /data/cora1/orion/nalu_build/install/trilinos/lib/libgaleri-xpetra.a /data/cora1/orion/nalu_build/install/trilinos/lib/libxpetra-sup.a /data/cora1/orion/nalu_build/install/trilinos/lib/libxpetra.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetraext.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetrainout.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetra.a /data/cora1/orion/nalu_build/install/trilinos/lib/libkokkoskernels.a /data/cora1/orion/nalu_build/install/trilinos/lib/libkokkosalgorithms.a /data/cora1/orion/nalu_build/install/trilinos/lib/libkokkoscontainers.a /data/cora1/orion/nalu_build/install/trilinos/lib/libkokkostsqr.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetraclassiclinalg.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetraclassicnodeapi.a /data/cora1/orion/nalu_build/install/trilinos/lib/libtpetraclassic.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchoskokkoscomm.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchoskokkoscompat.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchosremainder.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchosnumerics.a -llapack -lblas /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchoscomm.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchosparameterlist.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchosparser.a /data/cora1/orion/nalu_build/install/trilinos/lib/libteuchoscore.a /data/cora1/orion/nalu_build/install/trilinos/lib/libkokkoscore.a -ldl /data/cora1/orion/nalu_build/install/lib/libyaml-cpp.a -lmpi_usempi -lmpi_mpifh -lifport -lifcoremt -Wl,-rpath,/data/cora1/orion/nalu_build/install/lib:
ld: libnalu.a(H5IO.C.o): undefined reference to symbol 'H5check_version'
/opt/intel-2017/openmpi-2.1/hdf5-1.8.18/lib64/libhdf5.so.10: error adding symbols: DSO missing from command line

[jrood-nrel]

Looks like the HDF5 symbols are missing during the link process. Can we verify that Trilinos was built with HDF5 enabled? Or, rather, was https://github.com/NaluCFD/Nalu/blob/master/build/do-configTrilinos_release used to configure Trilinos? The CMake configure step in Nalu is set up to inherit the configuration of Trilinos, so we assume that HDF5 should be taken care of by Trilinos.

[opoplawski]

I used do-configTrilinos_release (modified as needed) to configure Trilinos, but it seems it did not enable HDF5 (at least not the HDF5 TPL):

Final set of enabled TPLs:  Pthread MPI BLAS LAPACK Boost Netcdf SuperLU BoostLib DLlib 9

Final set of non-enabled TPLs:  MKL yaml-cpp Peano CUDA CUSPARSE Thrust Cusp TBB HWLOC QTHREAD BinUtils ARPREC QD Scotch OVIS gpcd METIS MTMETIS ParMETIS PuLP TopoManager LibTopoMap PaToH CppUnit ADOLC ADIC TVMET MF ExodusII Nemesis XDMF Zlib HDF5 CGNS Pnetcdf y12m SuperLUDist SuperLUMT Cholmod UMFPACK MA28 AMD CSparse HYPRE PETSC BLACS SCALAPACK MUMPS PARDISO_MKL PARDISO Oski TAUCS ForUQTK Dakota HIPS MATLAB CASK SPARSKIT QT gtest BoostAlbLib OpenNURBS Portals CrayPortals Gemini InfiniBand BGPDCMF BGQPAMI Pablo HPCToolkit Clp GLPK qpOASES Matio PAPI MATLABLib Eigen X11 Lemon GLM quadmath CAMAL RTlib AmgX CGAL CGALCore VTune TASMANIAN ArrayFireCPU SimMesh SimModel SimParasolid SimAcis SimField Valgrind QUO 96

I also see:

CMake Warning:
  Manually-specified variables were not used by the project:

    HDF5_NO_SYSTEM_PATHS
    HDF5_ROOT

[jrood-nrel]

Ok I think that is the problem then. I'm a little worried about the version of Trilinos as well based on it ignoring HDF5_ROOT, since I believe specifying HDF5_ROOT as we currently do should work with the latest Trilinos develop branch. Using the <TPL>_ROOT hints in the CMake configure is something Trilinos enabled within the past year. So the next thing I would do is verify that you are using either Trilinos version 12.12.1, master branch, or the develop branch, which are the minimum versions required for the latest Nalu. Next, in the do-config script, you can try using:

-DHDF5_INCLUDE_DIRS:PATH=${hdf_install_dir}/include \
-DHDF5_LIBRARY_DIRS:PATH=${hdf_install_dir}/lib \

These more explicit hints can typically find TPLs in Trilinos where <TPL>_ROOT fails. Hopefully Trilinos will then find and build with HDF5 and subsequently, Nalu hopefully should as well.

[opoplawski]

I'm using the master branch of Trilinos at commit b7036615905c18ca108007c8803f71d8bae81aec

The main issues seems to be a missing "-DTPL_ENABLE_HDF5=ON" in do-configTrilinosrelease. After enabling that and setting the needed -DHDF5*_DIRS it found HDF5.

[jrood-nrel]

Thanks for looking into that. I have updated the config scripts to include TPL_ENABLE_HDF5.

[opoplawski]

thanks.