Closed shahzebsiddiqui closed 2 years ago
This covers the run of the e4s testsuite's gasnet, kokkos, superlu and superlu-dist tests on perlmutter, targeting e4s 21.11.
Gasnet test code failed to compile like:
cc `pkg-config --cflags gasnet-smp-par` -I. -c testbarrier.c
Package gasnet-smp-par was not found in the pkg-config search path.
Perhaps you should add the directory containing `gasnet-smp-par.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gasnet-smp-par' found
testbarrier.c:7:10: fatal error: gasnetex.h: No such file or directory
7 | #include <gasnetex.h>
| ^~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:24: testbarrier.o] Error 1
cc `pkg-config --cflags gasnet-smp-par` -I. -c testbarrier.c
Package gasnet-smp-par was not found in the pkg-config search path.
Perhaps you should add the directory containing `gasnet-smp-par.pc'
to the PKG_CONFIG_PATH environment variable
No package 'gasnet-smp-par' found
testbarrier.c:7:10: fatal error: gasnetex.h: No such file or directory
7 | #include <gasnetex.h>
| ^~~~~~~~~~~~
compilation terminated.
Kokkos test code cmake failed like:
-- Configuring done
CMake Error at CMakeLists.txt:31 (target_link_libraries):
Error evaluating generator expression:
$<LINK_LANGUAGE:CXX>
Expression did not evaluate to a known generator expression
CMake Error at CMakeLists.txt:35 (target_link_libraries):
Error evaluating generator expression:
$<LINK_LANGUAGE:CXX>
Expression did not evaluate to a known generator expression
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.
Superlu dist spack test failed like:
9 /usr/bin/ld: warning: libgfortran.so.5, needed by /opt/cray/pe/libsc
i/21.08.1.2/GNU/9.1/x86_64/lib/libsci_gnu.so, not found (try using -
rpath or -rpath-link)
10 /usr/bin/ld: warning: libfabric.so.1, needed by /opt/cray/pe/mpich/8
.1.13/ofi/gnu/9.1/lib/libmpi_gnu_91.so, not found (try using -rpath
or -rpath-link)
>> 11 /usr/bin/ld: /opt/cray/pe/libsci/21.08.1.2/GNU/9.1/x86_64/lib/libsci
_gnu.so: undefined reference to `_gfortran_concat_string@@GFORTRAN_8'
With numerous additional undefined symbols.
Todo: Investigate and correct any relevant problems with test construction. Determine if any package-side issues are still present as of the 22.05 e4s release and escalate as needed.
Thanks for looking into this. I just realized that it may have been good to post this issue directly in https://github.com/E4S-Project/testsuite instead of this project.
I don't think the testsuite repo should get bug reports unless they're with the functionality of the testsuite itself or they've already been determined to be failures caused by the construction of the test. In general issues identified by the testsuite should go to the e4s tracker for triage as you've done here.
The superlu-dist error seems to be fixed in 22.05 but another error in the package's test configuration was present. This PR should have it running: https://github.com/spack/spack/pull/30970
The gasnet and kokkos tests run successfully on the e4s 22.05 test deployment on perlmutter. I will check these again on the official NERSC 22.05 deployment when it becomes available. The superlu-dist test will fail for 22.05 unless we backport the above PR, once it's merged.
This issue is resolved by: https://github.com/E4S-Project/testsuite/commit/3d274f5e88e00bc81149c2fb19b71378bcfa66e2 https://software.nersc.gov/NERSC/buildtest-nersc/-/merge_requests/53 https://software.nersc.gov/NERSC/buildtest-nersc/-/merge_requests/61 https://software.nersc.gov/NERSC/buildtest-nersc/-/merge_requests/50
With this buildspec split into specs for the individual tests there may be new issues to report on a per-test basis.
Name of Software
testsuite
URL for Software
https://github.com/E4S-Project/testsuite
Contact Details
shahzebsiddiqui@lbl.gov
HPC System
Perlmutter
Request Description
We have this test https://software.nersc.gov/NERSC/buildtest-nersc/-/blob/devel/buildspecs/e4s/E4S-Testsuite/e4s_21.11_perlmutter.yml that we run against the Perlmutter E4S stack using buildtest. This runs several tests for different products and we noticed several errors that can be seen in CDASH https://my.cdash.org/test/56389573.
Relevant log output
Reproduce Bug
If you want to rerun the pipeline just go to https://software.nersc.gov/NERSC/buildtest-nersc/-/pipeline_schedules and press "Play" under Scheduled Pipeline