Amesos2: missing header causing Albany intel nightlies to fail

[ikalash]

It looks like there is a missing include in an Amesos2 file causing some of the Albany nightlies to fail: https://sems-cdash-son.sandia.gov/cdash/viewBuildError.php?buildid=83851

[TrilinosSerialInstall/include/Amesos2_CssMKL_FunctionMap.hpp:63](https://blob/master/TrilinosSerialInstall/include/Amesos2_CssMKL_FunctionMap.hpp#L63):10: fatal error: 'Amesos2_CssMKL_TypeMap.hpp' file not found

Curiously, it appears to be happening only using Intel compilers. Could someone please look into this?

@trilinos/amesos2

[ndellingwood]

@iyamazaki this is related to #13073 , can you look into this?

[iyamazaki]

@ndellingwood. I will look. It is strange that it cannot find the header file.

[ikalash]

Thank you! I am happy to share the modules / build scripts to reproduce the error. I find it strange that it only seems to happen with intel compiler.

[iyamazaki]

Hi @ikalash. Please let us know if #13111 has fixed the issue..

[ikalash]

@iyamazaki : thank you! I will check the nightlies tomorrow and will close this if the issue is fixed.

[ikalash]

Unfortunately now there are other linking related issues: https://sems-cdash-son.sandia.gov/cdash/viewBuildError.php?buildid=88148

[ 96%] Linking CXX executable StringUtils [ 96%] Linking CXX executable HessianVecFad /home/projects/albany/nightlyCDashTrilinosBlake/build-intel/TrilinosReleaseInstallIntel/lib64/libamesos2.so.15.2.0: undefined reference to `cluster_sparse_solver'

[iyamazaki]

Ah.. Thank you, @ikalash. We were wondering about this. At least for my version of MKL (older than yours..), Cluster Sparse Solver came with MKL, but it seems we need a separate TPL, @ndellingwood.

[ndellingwood]

@iyamazaki I didn't realize Pardiso was auto-magically enabled in Amesos2 when it is enabled as a TPL. I think the best route is to make the new CSS solver require an opt-in cmake option like some of the other solvers options (e.g. Amesos2_ENABLE_KLU2 etc) and default it to OFF

This will could be done by adding something along these lines:

1. A new variable like #cmakedefine HAVE_AMESOS2_PARDISO_CSS_MKL to cmake/Amesos2_config.h.in
2. An option to define the variable in CMakeLists.txt, but reuse the guards you already added to override and throw an error if dependencies are not met (IF (${PACKAGE_NAME}_ENABLE_PARDISO_MKL AND TPL_ENABLE_MPI)...); something like this in amesos2/CMakeLists.txt:

   IF (Amesos2_ENABLE_PARDISO_CSS_MKL)
   IF (NOT ${PACKAGE_NAME}_ENABLE_PARDISO_MKL OR NOT TPL_ENABLE_MPI)
   MESSAGE(FATAL "Amesos2_ENABLE_PARDISO_CSS_MKL requires PARDISO_MKL and MPI; these dependencies were not met. Please enable ... or disable Amesos2_ENABLE_PARDISO_CSS_MKL")
   ELSE()
   TRIBITS_ADD_OPTION_AND_DEFINE(${PACKAGE_NAME}_ENABLE_Pardiso_CSS_MKL
    HAVE_AMESOS2_PARDISO_CSS_MKL
    "Enable Pardiso_CSS_MKL in Amesos2"
    OFF
   )
   ENDIF()

3. Update the guard in src/CMakeLIsts.txt to use Amesos2_ENABLE_PARDISO_CSS_MKL
4. Update the src/Amesos2_Factory.* files to use HAVE_AMESOS2_PARDISO_CSS_MKL to guard the solver name options

There might be something cleaner, but I think something like the above is close to what would work. Are you able to test it out? I haven't built or used the Pardiso CSS stuff to confirm

[iyamazaki]

Yes, right now CSS is automatically enabled when Pardiso_MKL and MPI are both enabled. Just to make sure, can you tell me how Trilinos is configured, @ndellingwood, @ikalash?

[ikalash]

Sure @iyamazaki . Here are the Trilinos configure scripts for the 3 intel builds where we are seeing the problem:

• Blake: https://github.com/sandialabs/Albany/blob/master/doc/dashboards/blake.sandia.gov/do-cmake-trilinos-intel-release
• Attaway: https://github.com/sandialabs/Albany/blob/master/doc/dashboards/attaway-login10/do-cmake-trilinos-attaway-intel-serial
• CEE: https://github.com/sandialabs/Albany/blob/master/doc/dashboards/cee-compute011.sandia.gov/do-cmake-trilinos-mpi-sems-intel

[iyamazaki]

Hi, @ikalash. I hope the new PR fixed the issue, @ndellingwood..

[ikalash]

Thank you for fixing this! Unfortunately more Trilinos changes broke Albany during the past few days so I haven't been able to see if this particular issue is fixed. Once this other issue https://github.com/trilinos/Trilinos/pull/13155 is resolved, I should be able to see if this issue is fixed and close it if it is. Sorry for the delay!

[iyamazaki]

Thank you for reporting this, @ikalash!!