search

LLNL / serac

Serac is a high order nonlinear thermomechanical simulation code
BSD 3-Clause "New" or "Revised" License
178 stars 31 forks source link

Add nvToolsExt after all cuda libraries #1111

Closed white238 closed 3 months ago

white238 commented 3 months ago

@samuelpmishLLNL

For reasons, I don't understand but nvToolsExt needs to be after a lot of libraries otherwise you get missing symbols from the AMGX libraries.

codecov-commenter commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.58%. Comparing base (97fb1d7) to head (adcda2b).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #1111 +/- ## ======================================== Coverage 90.58% 90.58% ======================================== Files 160 160 Lines 14331 14331 ======================================== Hits 12982 12982 Misses 1349 1349 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

white238 commented 3 months ago

Another "fun" bit of info is the CUDA::nvToolsExt target from find_package(CUDAToolKit) does not include the actual library, even though it should have the shared library. I assume this has to do with the upcoming/current deprecation/moving to a header-only library for nvToolsExt but I needed the symbols for AMGx. This should have been in the upcoming CUDA::nvtx3 target. More info here:

https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html#nvtoolsext

samuelpmishLLNL commented 3 months ago

Is there a way to update AmgX's spack package so that the fix is not just specific to serac?

samuelpmishLLNL commented 3 months ago

here is the complete link line and error message when building with this branch:

/usr/bin/g++-12 -Wall -Wextra      -Wshadow -Wdouble-promotion -Wconversion -Wundef -Wnull-dereference -Wold-style-cast  -g -rdynamic    -fopenmp -L/usr/local/cuda/lib64 -Wl,-Bsymbolic-functions examples/simple_conduction/CMakeFiles/simple_conduction_without_input_file.dir/without_input_file.cpp.o CMakeFiles/simple_conduction_without_input_file.dir/cmake_device_link.o -o examples/simple_conduction_without_input_file -L/usr/local/cuda/targets/x86_64-linux/lib/stubs   -L/usr/local/cuda/targets/x86_64-linux/lib -Wl,-rpath,/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/serac-develop-arqj3wn7fh7kwfq4zqh56tj63ohratrb/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/serac-develop-arqj3wn7fh7kwfq4zqh56tj63ohratrb/lib64:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/amgx-2.3.0.1-bgapfufnnvyhomg2knyo24svuocelcnw/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/gcc-runtime-12.3.0-hio32wyqn5x36c23fldc5sxlcvkvsc6t/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/zlib-ng-2.1.6-lzstgsbitxrugm2veu3hsvu6knjqkuwd/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/lua-5.4.6-sapqeyah462eyfek57wphfah2ibs7xwv/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/readline-8.2-hppwzcut7bb54vqvelfnj4az2b6tcxnd/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/mfem-4.6.2.4-hgffss6egy6b2jb6zx43aqzfjuherw5d/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netcdf-c-4.7.4-s5yyz4ucrkdi4s3q2yodm4ux5oomjzmg/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/petsc-3.21.0-ptibifyjbacj32mrwrzsmuxzpvekgff3/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/strumpack-7.2.0-tmcxhaoexzq7stfxcs6qund7k5j4tx7y/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netlib-scalapack-2.2.0-qcuv2i6kxjkqw7mtoehmptoaobzty3ss/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/sundials-6.7.0-sxcdkg2pvyhiawtf2tcy2u5gtawjiqxy/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/raja-2024.02.0-2jc6syt3fqeovfjljuwlfivedbkvxg3u/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/camp-2024.02.0-o2xfmsdbh5ojxuosjqnldypfwwdlzha4/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/umpire-2024.02.0-dw7tediw4pw3v5b2kcbwufoolcwzsw2n/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/fmt-10.2.1-qug2m6e5qfdo2lya2nbelsclw6qrqxp4/lib:/home/sam/.local/lib:/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/tribol-0.1.0.15-jgh4rgy2b2wbx55fa2d4uwlqas4s7irv/lib:/usr/local/cuda/lib64  lib/libserac_physics.a  lib/libserac_mesh.a  lib/libserac_functional.a  lib/libserac_boundary_conditions.a  lib/libserac_state.a  lib/libserac_contact.a  lib/libserac_numerics.a  lib/libserac_physics_materials.a  lib/libserac_functional.a  lib/libserac_mesh.a  lib/libserac_infrastructure.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/tribol-0.1.0.15-jgh4rgy2b2wbx55fa2d4uwlqas4s7irv/lib/libtribol.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/tribol-0.1.0.15-jgh4rgy2b2wbx55fa2d4uwlqas4s7irv/lib/libredecomp.a  -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lHYPRE -lopenblas -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -lsuperlu_dist -lparmetis -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lopenblas -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -lmetis -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lopenblas -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/sundials-6.7.0-sxcdkg2pvyhiawtf2tcy2u5gtawjiqxy/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/sundials-6.7.0-sxcdkg2pvyhiawtf2tcy2u5gtawjiqxy/lib -lsundials_arkode -lsundials_cvodes -lsundials_nvecserial -lsundials_kinsol -lsundials_nvecparallel -lsundials_nvecmpiplusx -lsundials_nveccuda -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/strumpack-7.2.0-tmcxhaoexzq7stfxcs6qund7k5j4tx7y/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/strumpack-7.2.0-tmcxhaoexzq7stfxcs6qund7k5j4tx7y/lib -lstrumpack -Xlinker -rpath -Xlinker /usr/lib/gcc/x86_64-linux-gnu/12 -L/usr/lib/gcc/x86_64-linux-gnu/12 -lgfortran -lmpifort -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -lparmetis -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netlib-scalapack-2.2.0-qcuv2i6kxjkqw7mtoehmptoaobzty3ss/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netlib-scalapack-2.2.0-qcuv2i6kxjkqw7mtoehmptoaobzty3ss/lib -lscalapack -lcusolver -lcublas -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netcdf-c-4.7.4-s5yyz4ucrkdi4s3q2yodm4ux5oomjzmg/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netcdf-c-4.7.4-s5yyz4ucrkdi4s3q2yodm4ux5oomjzmg/lib -lnetcdf -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib -lhdf5_hl -lhdf5 -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/petsc-3.21.0-ptibifyjbacj32mrwrzsmuxzpvekgff3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/petsc-3.21.0-ptibifyjbacj32mrwrzsmuxzpvekgff3/lib -lpetsc -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -Xlinker -rpath -Xlinker /usr/lib/gcc/x86_64-linux-gnu/12 -L/usr/lib/gcc/x86_64-linux-gnu/12 -lHYPRE -lsuperlu_dist -lopenblas -lparmetis -lmetis -lm -lmpichfort -lmpich -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath -lcusparse -lcusolver -lcublas -lnvToolsExt -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/amgx-2.3.0.1-bgapfufnnvyhomg2knyo24svuocelcnw/lib -lamgx -lcusparse -lrt -Xlinker -rpath -Xlinker /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/zlib-ng-2.1.6-lzstgsbitxrugm2veu3hsvu6knjqkuwd/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/zlib-ng-2.1.6-lzstgsbitxrugm2veu3hsvu6knjqkuwd/lib -lz  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_quest.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_slam.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_mint.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_klee.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_inlet.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_sidre.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_slic.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_lumberjack.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/axom-0.9.0.1-xedxtprfh42igx27dsyadyp2o2mg4mjs/lib/libaxom_core.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/umpire-2024.02.0-dw7tediw4pw3v5b2kcbwufoolcwzsw2n/lib/libumpire.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/fmt-10.2.1-qug2m6e5qfdo2lya2nbelsclw6qrqxp4/lib/libfmt.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/raja-2024.02.0-2jc6syt3fqeovfjljuwlfivedbkvxg3u/lib/libRAJA.a  -lnvToolsExt  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/camp-2024.02.0-o2xfmsdbh5ojxuosjqnldypfwwdlzha4/lib/libcamp.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit_relay_mpi_io.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit_relay.a  -lrt  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib/libhdf5.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit_blueprint_mpi.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit_blueprint.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit_relay_mpi.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/conduit-0.9.1-fdle5ykhzyve3ltgg4kyqqieenipeamf/lib/libconduit.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib/libparmetis.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib/libmetis.a  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib/libhdf5.a  -lm  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/zlib-ng-2.1.6-lzstgsbitxrugm2veu3hsvu6knjqkuwd/lib/libz.so  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/lua-5.4.6-sapqeyah462eyfek57wphfah2ibs7xwv/lib/liblua54.so  -lm  /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/mfem-4.6.2.4-hgffss6egy6b2jb6zx43aqzfjuherw5d/lib/libmfem.a  /usr/local/cuda/lib64/libcudart_static.a  -ldl  /usr/lib/x86_64-linux-gnu/librt.a  -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lHYPRE -lopenblas -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -lsuperlu_dist -lparmetis -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lopenblas -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -lmetis -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -lopenblas -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/sundials-6.7.0-sxcdkg2pvyhiawtf2tcy2u5gtawjiqxy/lib -lsundials_arkode -lsundials_cvodes -lsundials_nvecserial -lsundials_kinsol -lsundials_nvecparallel -lsundials_nvecmpiplusx -lsundials_nveccuda -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/strumpack-7.2.0-tmcxhaoexzq7stfxcs6qund7k5j4tx7y/lib -lstrumpack -L/usr/lib/gcc/x86_64-linux-gnu/12 -lgfortran -lmpifort -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -lparmetis -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netlib-scalapack-2.2.0-qcuv2i6kxjkqw7mtoehmptoaobzty3ss/lib -lscalapack -lcusolver -lcublas -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/netcdf-c-4.7.4-s5yyz4ucrkdi4s3q2yodm4ux5oomjzmg/lib -lnetcdf -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hdf5-1.8.23-wuxgpyfxukz4hy6mrs2wulptpihvtfad/lib -lhdf5_hl -lhdf5 -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/petsc-3.21.0-ptibifyjbacj32mrwrzsmuxzpvekgff3/lib -lpetsc -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/hypre-2.26.0-ncfa2miwza7t7a4gadov6hyxi2z24akx/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/superlu-dist-8.1.2-fuk2t2gyjpag5rljesc4akeg4l3hoooj/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/openblas-0.3.26-wfq52jjryfenk54kf3qqgndee7vgmsh3/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/parmetis-4.0.3-tdfzc3tyxr6to6j2rdt3eiiuoh6gykpy/lib -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/metis-5.1.0-my43hh2r636htpgacvxd7i3wgswb7a4d/lib -L/usr/lib/gcc/x86_64-linux-gnu/12 -lHYPRE -lsuperlu_dist -lopenblas -lparmetis -lmetis -lm -lmpichfort -lmpich -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lstdc++ -lquadmath -lcusparse -lcusolver -lcublas -lnvToolsExt -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/amgx-2.3.0.1-bgapfufnnvyhomg2knyo24svuocelcnw/lib -lamgx -lcusparse -lrt -L/home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/zlib-ng-2.1.6-lzstgsbitxrugm2veu3hsvu6knjqkuwd/lib -lz  -L/usr/local/cuda/lib64  /usr/local/cuda/lib64/libcublas.so  /usr/local/cuda/lib64/libcusolver.so  /usr/lib/x86_64-linux-gnu/libmpich.so  /usr/lib/x86_64-linux-gnu/libmpichcxx.so  -lcudadevrt  -lcudart_static  -lrt  -lpthread  -ldl && :
[build] /usr/bin/ld: /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/amgx-2.3.0.1-bgapfufnnvyhomg2knyo24svuocelcnw/lib/libamgx.a(amgx_timer.cu.o): in function `amgx::nvtxRange::nvtxRange(char const*, int)':
[build] tmpxft_0025de4d_00000000-6_amgx_timer.cudafe1.cpp:(.text+0xae): undefined reference to `nvtxRangePushEx'
[build] /usr/bin/ld: /home/sam/code/serac_libs_cuda_4_30_2024/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-12.3.0/amgx-2.3.0.1-bgapfufnnvyhomg2knyo24svuocelcnw/lib/libamgx.a(amgx_timer.cu.o): in function `amgx::nvtxRange::~nvtxRange()':
[build] tmpxft_0025de4d_00000000-6_amgx_timer.cudafe1.cpp:(.text+0xd9): undefined reference to `nvtxRangePop'

I also tried rebuilding the TPLs again from scratch after seeing this error, and it didn't seem to make a difference.

note: it seems to only be the simple_conduction_without_input_file target that is failing with this error message

white238 commented 3 months ago

Is there a way to update AmgX's spack package so that the fix is not just specific to serac?

I have a meeting with a Spack guy in a bit, I will talk to him to see if we can get it in the appropriate spot. I glanced over it at one point and it was a mild mystery where things were coming from.