search

trilinos / Trilinos

Primary repository for the Trilinos Project
https://trilinos.org/
Other
1.22k stars 570 forks source link

Teuchos: Cannot build on vortex v100 debug #5285

Closed tcfisher closed 4 years ago

tcfisher commented 5 years ago

Bug Report

@trilinos/teuchos

Description

I get the following errors when building Trilinos using the Sparc ats2-v100 configuration and cuda-xlc with debug.

    1500-004: (U) INTERNAL COMPILER ERROR while compiling Teuchos::close(Teuchos::StateInProgress &, const Configs &, const Teuchos::Grammar &, const Gr
aph &).  Compilation ended.  Contact your Service Representative and provide the following information: GRARNN: gr16700 is used before it is defined. Fo
r more information visit: http://www.ibm.com/support/docview.wss?uid=swg21110810
1586-346 (U) An error occurred during code generation.  The code generation return code was 1.
make[2]: *** [packages/teuchos/parser/src/CMakeFiles/teuchosparser.dir/build.make:63: packages/teuchos/parser/src/CMakeFiles/teuchosparser.dir/Teuchos_make_lalr1_parser.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1870: packages/teuchos/parser/src/CMakeFiles/teuchosparser.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Steps to Reproduce

  1. SHA1: a2cf1edb663fb9ab0309ef977230949b6c5e299
  2. Configure script: see sparc Trilinos repo

@bartlettroscoe who to contact?

jhux2 commented 5 years ago

@trilinos/teuchos @nmhamster

nmhamster commented 5 years ago

@tcfisher - you won the lottery ticket! Can you show me the compile line (make VERBOSE=1)?

tcfisher commented 5 years ago

@nmhamster Thank you. I feel very fortunate. 

cd /ascldap/users/sparc/Trilinos/ats2-v100_xl-2019.02.07_cuda-9.2.148_spmpi-2019.01.30_static_dbg_build/packages/teuchos/parser/src && /usr/tce/packages/spectrum-mpi/spectrum-mpi-2019.01.30-xl-2019.02.07/bin/mpicxx -+ -DTEUCHOSPARSER_LIB_EXPORTS_MODE -I/ascldap/users/sparc/Trilinos/ats2-v100_xl-2019.02.07_cuda-9.2.148_spmpi-2019.01.30_static_dbg_build -I/ascldap/users/sparc/Trilinos/packages/teuchos/parser/src -I/ascldap/users/sparc/Trilinos/ats2-v100_xl-2019.02.07_cuda-9.2.148_spmpi-2019.01.30_static_dbg_build/packages/teuchos/core/src -I/ascldap/users/sparc/Trilinos/packages/teuchos/core/src -I/ascldap/users/sparc/Trilinos/ats2-v100_xl-2019.02.07_cuda-9.2.148_spmpi-2019.01.30_static_dbg_build/packages/kokkos/core/src -I/ascldap/users/sparc/Trilinos/packages/kokkos/core/src -I/usr/tce/packages/cuda/cuda-9.2.148/include -I/usr/gapps/sparc/tpls/ats2-v100/boost-1.65.1/ats2-v100_xl-2019.02.07_cuda-9.2.148/include  -lineinfo -ccbin xlC -O3  --std=c++11 -lineinfo -g -Xcudafe --diag_suppress=esa_on_defaulted_function_ignored -expt-extended-lambda -arch=sm_70   -g   -o CMakeFiles/teuchosparser.dir/Teuchos_make_lalr1_parser.cpp.o -c /ascldap/users/sparc/Trilinos/packages/teuchos/parser/src/Teuchos_make_lalr1_parser.cpp
nmhamster commented 5 years ago

@tcfisher - is this Waterman, Vortex or something else?

tcfisher commented 5 years ago

@nmhamster this is vortex. waterman didn't seem to have any issues.

nmhamster commented 5 years ago

@tcfisher - these are a tiny version apart (compiler version) which could be the source of the new problem. I'll have to take a look. This is super urgent?

tcfisher commented 5 years ago

@nmhamster This is not urgent. We just wanted to raise awareness now to try to get a solution in place before we absolutely need it.

nmhamster commented 5 years ago

@tcfisher / @bartlettroscoe - talked a lot more with IBM this morning. They are sending some additional details in how to diagnose these problems in field. Hope to use these flags for debugging etc and work on required fixes. Just wanted to keep you in the loop.

bartlettroscoe commented 5 years ago

I think @ibaned wrote this code?

ibaned commented 5 years ago

Yes this is from the generalized parser I wrote... not sure what aspect of it caused an ICE in IBM's compiler.... the C++ is not too exotic I think...

nmhamster commented 5 years ago

Keep calm. We are working on some of these issues with IBM and have a path forward on a few bugs.

bartlettroscoe commented 4 years ago

@tcfisher, is this still active? SPARC has an XL build using Trilinos on ATS-2 ('vortex'), no?

tcfisher commented 4 years ago

@bartlettroscoe, I will get back to you. We have nightly build and test of release with cuda-xl, but I'm unsure if we have debug.

tcfisher commented 4 years ago

@bartlettroscoe It looks like our debug build worked fine. You can close this out.

bartlettroscoe commented 4 years ago

@bartlettroscoe It looks like our debug build worked fine. You can close this out.

Thanks! Closing as complete.