Stratimikos: Compiling Trilinos on OLCF's Summit

[kgottiparthi]

I am trying to compile Trilinos on OLCF's summit. I have the following in my configure file. Specifically I need Thyra_AztecOOLinearOpWithSolveFactory.hpp header file to compile my Trilinos based code. Although I turn on both Stratimikos and AztecOO in my configure file I still donot get the required header file. Please inform me if I am missing anything.

Thank you, Kalyan

!/bin/bash

cmake \ -D CMAKE_BUILD_TYPE:STRING=RELEASE \ -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \ -D TPL_ENABLE_Netcdf:BOOL=ON \ -D TPL_ENABLE_Pnetcdf:STRING=ON \ -D Trilinos_ENABLE_Komplex=OFF \ -D EpetraExt_USING_HDF5=OFF \ -D Trilinos_ENABLE_Kokkos=ON \ -D Tpetra_INST_SERIAL=ON \ -D Tpetra_INST_CUDA:BOOL=ON \ -D Tpetra_INST_INT_INT:BOOL=OFF \ -D Tpetra_INST_INT_LONG:BOOL=ON \ -D Tpetra_INST_INT_LONG_LONG:BOOL=OFF \ -D TPL_ENABLE_CUDA=ON \ -D Kokkos_ENABLE_CUDA:BOOL=ON \ -D Kokkos_ENABLE_CUDA_UVM:BOOL=OFF \ -D Kokkos_ARCH_VOLTA70=ON \ -D Kokkos_ARCH_POWER9=ON \ -D Kokkos_ENABLE_PTHREAD=OFF \ -D Kokkos_ENABLE_SERIAL=ON \ -D Kokkos_ENABLE_CUDA_LAMBDA=ON \ -D TPL_ENABLE_HDF5:STRING=ON \ -D HDF5_INCLUDE_DIRS:PATH="/sw/summit/spack-envs/summit-plus/opt/gcc-12.1.0/hdf5-1.14.3-qkrpzuwj32v6awqw5rgv5g5kyurnt563/include" \ -D HDF5_LIBRARY_DIRS:PATH="/sw/summit/spack-envs/summit-plus/opt/gcc-12.1.0/hdf5-1.14.3-qkrpzuwj32v6awqw5rgv5g5kyurnt563/lib" \ -D Trilinos_ENABLE_OpenMP:BOOL=OFF \ -D TPL_ENABLE_MPI:BOOL=ON \ -D MPI_BASE_DIR:PATH="/sw/summit/spack-envs/summit-plus/opt/gcc-12.1.0/spectrum-mpi-10.4.0.6-20230210-db5xakaaqowbhp3nqwebpxrdbwtm4knu" \ -D Trilinos_ENABLE_NOX:BOOL=ON \ -D Trilinos_ENABLE_ALL_PACKAGES=ON \ -D Trilinos_ENABLE_ShyLU:STRING="OFF" \ -D Trilinos_ENABLE_ShyLU_NodeTacho=OFF \ -D Trilinos_ENABLE_Teko=OFF \ -D Trilinos_ENABLE_Shards=OFF \ -D Trilinos_ENABLE_Panzer=ON \ -D Trilinos_ENABLE_Sacado=ON \ -D Trilinos_ENABLE_STKMesh=ON \ -D Trilinos_ENABLE_STKIO=ON \ -D Trilinos_ENABLE_SEACAS=ON \ -D Trilinos_ENABLE_Zoltan2=ON \ -D Trilinos_ENABLE_Shards=ON \ -D Trilinos_ENABLE_Intrepid2=ON \ -D Trilinos_ENABLE_Tpetra=ON \ -D Trilinos_ENABLE_Epetra=ON \ -D Trilinos_ENABLE_PanzerDofMgr=ON \ -D Tpetra_INST_COMPLEX_DOUBLE=ON \ -D Tpetra_INST_COMPLEX_FLOAT=OFF \ -D Tpetra_INST_FLOAT=OFF \ -D Teuchos_ENABLE_FLOAT=OFF \ -D Tpetra_INST_SERIAL=ON \ -D Tpetra_INST_OPENMP:BOOL=OFF \ -D Tpetra_INST_CUDA=ON \ -D Trilinos_ENABLE_COMPLEX=ON \ -D Trilinos_ENABLE_Belos=ON \ -D Trilinos_ENABLE_Ifpack2=ON \ -D Trilinos_ENABLE_Amesos2=ON \ -D Trilinos_ENABLE_MueLu=ON \ -D Trilinos_ENABLE_Stratimikos=ON \ -D Trilinos_ENABLE_ML=ON \ -D Trilinos_ENABLE_NOX=ON \ -D Trilinos_ENABLE_Tempus=ON \ -D Trilinos_ENABLE_Panzer=ON \ -D Trilinos_ENABLE_AztecOO=ON \ -D TPL_ENABLE_BLAS=ON \ -D TPL_ENABLE_LAPACK=ON \ -D TPL_ENABLE_HDF5=ON \ -D TPL_ENABLE_Netcdf=ON \ -D TPL_ENABLE_CGNS=OFF \ -D TPL_ENABLE_ParMETIS=OFF \ -D TPL_ENABLE_Matio=OFF \ -D TPL_ENABLE_Krino=OFF \ -D TPL_ENABLE_X11=OFF \ -D TPL_ENABLE_gtest=OFF \ -D TPL_ENABLE_Gtest=OFF \ -D TPL_ENABLE_PyTrilinos=OFF \ -D BUILD_SHARED_LIBS=ON \ -D Trilinos_ENABLE_Fortran:BOOL=OFF \ -D Tpetra_ASSUME_CUDA_AWARE_MPI=OFF \ -D Trilinos_ENABLE_SEACASAlgebra:STRING=ON \ -D Trilinos_ENABLE_SEACASExotxt:STRING=ON \ -D Trilinos_ENABLE_SEACASExodiff:STRING=ON \ -D Trilinos_ENABLE_SEACASNemesis:STRING=ON \ -D Trilinos_ENABLE_SEACASNemslice:STRING=ON \ -D Trilinos_ENABLE_SEACASNemspread:STRING=ON \ -D Trilinos_ENABLE_SEACASChaco:STRING=ON \ -D Trilinos_ENABLE_SEACASConjoin:STRING=ON \ -D Trilinos_ENABLE_SEACASEjoin:STRING=ON \ -D Trilinos_ENABLE_SEACASGjoin:STRING=ON \ -D Trilinos_ENABLE_SEACASEpu:STRING=ON \ -D Trilinos_ENABLE_SEACASSuplib:STRING=ON \ -D Trilinos_ENABLE_Zoltan:STRING="ON" \ -D Trilinos_ENABLE_MueLu:STRING=ON \ -D Trilinos_ENABLE_ML:STRING=ON \ -D TPL_ENABLE_X11=OFF \ -D Trilinos_ENABLE_EpetraExt=OFF \ -D Kokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE=OFF \ -D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \ -D CMAKE_INSTALL_PREFIX:PATH=${TRILINOS_ROOT} \ -D CMAKE_CXX_STANDARD=17 \ ${BASE_DIR}/${TRILINOS_DIRNAME}

[cgcgcg]

Looks like you need EpetraExt: https://github.com/trilinos/Trilinos/blob/77005adad6d625dbf62009620ffdc4ffa06b9fac/packages/stratimikos/adapters/CMakeLists.txt#L22-L24

Maybe this is a silly question: why are you compiling CPU-only legacy code on summit?

[kgottiparthi]

Thank you. I will turn on EpetraExt. We want both CPU and GPU version for a project. How may I turn on GPU offloading?

[cgcgcg]

That's my point. Epetra is a legacy package that has no GPU capabilities and it will get removed from Trilinos in the future. You want to switch to Tpetra which supports CPU and GPU.

[kgottiparthi]

Thank you. I will inform the dev team to deprecate Epetra.