search

pghysels / STRUMPACK

Structured Matrix Package (LBNL)
http://portal.nersc.gov/project/sparse/strumpack/
Other
167 stars 41 forks source link

Make CUDA and MPI exclusive #32

Open nindanaoto opened 4 years ago

nindanaoto commented 4 years ago

Hello, maintainers.

I'm currently trying to integrate STRUMPACK CUDA support into PETSc and found that STRUMPACK_USE_MPI and STRUMPACK_USE_CUDA cannot be switched ON at the same time because nvcc can't process mpi.h which is included in /src/misc/Triplet.hpp.

I think these options should be handled as exclusive ones in CMakeLists.txt.

pghysels commented 4 years ago

I haven't seen that problem before. Does it say it cannot find mpi.h or is there another error? What options are you passing to CMake? Are setting the compilers to the MPI wrappers when running cmake? It should not be necessary to do that. CMake should find the MPI libraries and headers.

nindanaoto commented 4 years ago

Thank you for your quick reply!

The following is the related log when I tried to compile STRUMPACK in PETSc. I'm sorry for I didn't post this first.

As you say, CMake does find MPI, but looking at the end of the log, it looks like nvcc doesn't handle MPI well.

                        Configuring STRUMPACK with cmake; this may take several minutes
                  =============================================================================================

Executing: /usr/bin/cmake .. -DCMAKE_INSTALL_PREFIX=/home/nimda/sources/petsc/real_strumpack_cuda -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_C_COMPILER="mpicc" -DCMAKE_AR=/usr/bin/ar -DCMAKE_RANLIB=/usr/bin/ranlib -DCMAKE_C_FLAGS:STRING="-fstack-protector -O3 -march=native" -DCMAKE_C_FLAGS_DEBUG:STRING="-fstack-protector -O3 -march=native" -DCMAKE_C_FLAGS_RELEASE:STRING="-fstack-protector -O3 -march=native" -DCMAKE_CXX_COMPILER="mpicxx" -DCMAKE_CXX_FLAGS:STRING="-fstack-protector -O3 -march=native" -DCMAKE_CXX_FLAGS_DEBUG:STRING="-fstack-protector -O3 -march=native" -DCMAKE_CXX_FLAGS_RELEASE:STRING="-fstack-protector -O3 -march=native" -DCMAKE_Fortran_COMPILER="mpif90" -DCMAKE_Fortran_FLAGS:STRING="-ffree-line-length-0 -O3 -march=native" -DCMAKE_Fortran_FLAGS_DEBUG:STRING="-ffree-line-length-0 -O3 -march=native" -DCMAKE_Fortran_FLAGS_RELEASE:STRING="-ffree-line-length-0 -O3 -march=native" -DBUILD_SHARED_LIBS=off -DTPL_BLAS_LIBRARIES="-llapack -lblis -lm -lstdc++ -ldl -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lgfortran -lm -lgcc_s -lquadmath -lpthread" -DTPL_LAPACK_LIBRARIES="-llapack -lblis -lm -lstdc++ -ldl -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lgfortran -lm -lgcc_s -lquadmath -lpthread" -DTPL_METIS_LIBRARIES="-Wl,-rpath,/home/nimda/sources/petsc/real_strumpack_cuda/lib -L/home/nimda/sources/petsc/real_strumpack_cuda/lib -lmetis" -DTPL_METIS_INCLUDE_DIRS="/home/nimda/sources/petsc/real_strumpack_cuda/include" -DTPL_ENABLE_PARMETIS=ON -DTPL_PARMETIS_LIBRARIES="-Wl,-rpath,/home/nimda/sources/petsc/real_strumpack_cuda/lib -L/home/nimda/sources/petsc/real_strumpack_cuda/lib -lparmetis" -DTPL_PARMETIS_INCLUDE_DIRS="/home/nimda/sources/petsc/real_strumpack_cuda/include" -DTPL_ENABLE_SCOTCH=OFF -DCMAKE_BUILD_TYPE=Release -DSTRUMPACK_USE_OPENMP=OFF -DSTRUMPACK_USE_MPI=ON -DSTRUMPACK_USE_CUDA=ON -DMPI_C_COMPILER="mpicc" -DMPI_CXX_COMPILER="mpicxx" -DMPI_Fortran_COMPILER="mpif90" -DCMAKE_INSTALL_NAME_DIR:STRING="/home/nimda/sources/petsc/real_strumpack_cuda/lib" stdout: -- The CXX compiler identification is GNU 9.3.0 -- The C compiler identification is GNU 9.3.0 -- The Fortran compiler identification is GNU 9.3.0 -- Check for working CXX compiler: /usr/bin/mpicxx -- Check for working CXX compiler: /usr/bin/mpicxx -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Check for working C compiler: /usr/bin/mpicc -- Check for working C compiler: /usr/bin/mpicc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working Fortran compiler: /usr/bin/mpif90 -- Check for working Fortran compiler: /usr/bin/mpif90 -- works -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Checking whether /usr/bin/mpif90 supports Fortran 90 -- Checking whether /usr/bin/mpif90 supports Fortran 90 -- yes -- Detecting Fortran/C Interface -- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling -- Verifying Fortran/CXX Compiler Compatibility -- Verifying Fortran/CXX Compiler Compatibility - Success -- Found MPI_C: /usr/bin/mpicc (found version "3.1") -- Found MPI_CXX: /usr/bin/mpicxx (found version "3.1") -- Found MPI_Fortran: /usr/bin/mpif90 (found version "3.1") -- Found MPI: TRUE (found version "3.1")
-- OpenMP support was not enabled. -- Looking for a CUDA compiler -- Looking for a CUDA compiler - /usr/local/cuda-11.1/bin/nvcc -- The CUDA compiler identification is NVIDIA 11.1.74 -- Check for working CUDA compiler: /usr/local/cuda-11.1/bin/nvcc -- Check for working CUDA compiler: /usr/local/cuda-11.1/bin/nvcc -- works -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Found CUDAToolkit: /usr/local/cuda-11.1/include (found version "10.1.243") -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE
-- Using BLAS from TPL_BLAS_LIBRARIES (-llapack -lblis -lm -lstdc++ -ldl -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lgfortran -lm -lgcc_s -lquadmath -lpthread) -- Using LAPACK from TPL_LAPACK_LIBRARIES (-llapack -lblis -lm -lstdc++ -ldl -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi -lgfortran -lm -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lgfortran -lm -lgcc_s -lquadmath -lpthread) -- Performing Test STRUMPACK_USE_GETOPT -- Performing Test STRUMPACK_USE_GETOPT - Success -- Found SCALAPACK scalapack -- Found METIS: -Wl,-rpath,/home/nimda/sources/petsc/real_strumpack_cuda/lib -L/home/nimda/sources/petsc/real_strumpack_cuda/lib -lmetis (found version "5.1.0")
-- Could NOT find SCOTCH (missing: SCOTCH_LIBRARIES SCOTCH_INCLUDE_DIR) -- Found ParMETIS: -Wl,-rpath,/home/nimda/sources/petsc/real_strumpack_cuda/lib -L/home/nimda/sources/petsc/real_strumpack_cuda/lib -lparmetis
-- Performing Test COMPILER_SUPPORTS_NOOVERLOAD_WARNING -- Performing Test COMPILER_SUPPORTS_NOOVERLOAD_WARNING - Success -- Performing Test COMPILER_SUPPORTS_NOPRAGMA_WARNING -- Performing Test COMPILER_SUPPORTS_NOPRAGMA_WARNING - Success -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) -- Configuring done -- Generating done -- Build files have been written to: /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build

                        Compiling and installing STRUMPACK; this may take several minutes
                  =============================================================================================

Executing: /usr/bin/make -j13 -l24.0 stdout: /usr/bin/cmake -S/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack -B/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles/progress.marks /usr/bin/make -f CMakeFiles/Makefile2 all make[1]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' /usr/bin/make -f CMakeFiles/strumpack.dir/build.make CMakeFiles/strumpack.dir/depend make[2]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' cd /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles/strumpack.dir/DependInfo.cmake --color= Scanning dependencies of target strumpack make[2]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' /usr/bin/make -f CMakeFiles/strumpack.dir/build.make CMakeFiles/strumpack.dir/build make[2]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' [ 1%] Building CXX object CMakeFiles/strumpack.dir/src/misc/Triplet.cpp.o [ 1%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverC.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/misc/Triplet.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/misc/Triplet.cpp [ 1%] Building Fortran object CMakeFiles/strumpack.dir/src/fortran/strumpack.f90.o [ 2%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CompressedSparseMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverC.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverC.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CompressedSparseMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CompressedSparseMatrix.cpp /usr/bin/mpif90 -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -I/home/nimda/sources/petsc/real_strumpack_cuda/include -I/usr/local/cuda-11.1/include -ffree-line-length-0 -O3 -march=native -ffree-line-length-0 -O3 -march=native -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/fortran/strumpack.f90 -o CMakeFiles/strumpack.dir/src/fortran/strumpack.f90.o [ 2%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackOptions.cpp.o [ 4%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackParameters.cpp.o [ 5%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverMPIDist.cpp.o [ 6%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolver.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackOptions.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackOptions.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverMPIDist.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverMPIDist.cpp [ 8%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverBase.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackParameters.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackParameters.cpp [ 8%] Building C object CMakeFiles/strumpack.dir/src/sparse/strumpack_mc64ad.c.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolver.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolver.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverBase.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverBase.cpp /usr/bin/mpicc -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -o CMakeFiles/strumpack.dir/src/sparse/strumpack_mc64ad.c.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/strumpack_mc64ad.c [ 8%] Building CXX object CMakeFiles/strumpack.dir/src/misc/TaskTimer.cpp.o [ 9%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRGraph.cpp.o [ 9%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/misc/TaskTimer.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/misc/TaskTimer.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRGraph.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRGraph.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRMatrix.cpp [ 10%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTree.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTree.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTree.cpp [ 10%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/SeparatorTree.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/SeparatorTree.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/SeparatorTree.cpp [ 12%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRMatrixMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRMatrixMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRMatrixMPI.cpp [ 12%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTreeMPI.cpp [ 13%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPIDist.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPIDist.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTreeMPIDist.cpp [ 13%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/PropMapSparseMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/PropMapSparseMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/PropMapSparseMatrix.cpp [ 14%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStab.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStab.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/BiCGStab.cpp [ 16%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/GMRes.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/GMRes.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/GMRes.cpp [ 16%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinement.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinement.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/IterativeRefinement.cpp [ 17%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/GMResMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/GMResMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/GMResMPI.cpp [ 17%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStabMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStabMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/BiCGStabMPI.cpp [ 18%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinementMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinementMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/IterativeRefinementMPI.cpp [ 18%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontFactory.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontFactory.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontFactory.cpp [ 20%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrix.cpp [ 21%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDense.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDense.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixDense.cpp [ 21%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSS.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSS.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixHSS.cpp [ 22%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixMPI.cpp [ 22%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDenseMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDenseMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixDenseMPI.cpp [ 24%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSSMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSSMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixHSSMPI.cpp [ 24%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixBLRMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixBLRMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixBLRMPI.cpp [ 25%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/ExtendAdd.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/ExtendAdd.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/ExtendAdd.cpp [ 25%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixGPU.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixGPU.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixGPU.cpp [ 26%] Building CUDA object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixCUDA.cu.o /usr/local/cuda-11.1/bin/nvcc -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem=/home/nimda/sources/petsc/real_strumpack_cuda/include -O3 -DNDEBUG -std=c++11 -x cu -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixCUDA.cu -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixCUDA.cu.o make[2]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' make[1]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' Error running make on STRUMPACK: Could not execute "['/usr/bin/make -j13 -l24.0 ']": /usr/bin/cmake -S/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack -B/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles/progress.marks /usr/bin/make -f CMakeFiles/Makefile2 all make[1]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' /usr/bin/make -f CMakeFiles/strumpack.dir/build.make CMakeFiles/strumpack.dir/depend make[2]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' cd /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build/CMakeFiles/strumpack.dir/DependInfo.cmake --color= Scanning dependencies of target strumpack make[2]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' /usr/bin/make -f CMakeFiles/strumpack.dir/build.make CMakeFiles/strumpack.dir/build make[2]: Entering directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' [ 1%] Building CXX object CMakeFiles/strumpack.dir/src/misc/Triplet.cpp.o [ 1%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverC.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/misc/Triplet.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/misc/Triplet.cpp [ 1%] Building Fortran object CMakeFiles/strumpack.dir/src/fortran/strumpack.f90.o [ 2%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CompressedSparseMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverC.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverC.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CompressedSparseMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CompressedSparseMatrix.cpp /usr/bin/mpif90 -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -I/home/nimda/sources/petsc/real_strumpack_cuda/include -I/usr/local/cuda-11.1/include -ffree-line-length-0 -O3 -march=native -ffree-line-length-0 -O3 -march=native -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/fortran/strumpack.f90 -o CMakeFiles/strumpack.dir/src/fortran/strumpack.f90.o [ 2%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackOptions.cpp.o [ 4%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackParameters.cpp.o [ 5%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverMPIDist.cpp.o [ 6%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolver.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackOptions.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackOptions.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverMPIDist.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverMPIDist.cpp [ 8%] Building CXX object CMakeFiles/strumpack.dir/src/StrumpackSparseSolverBase.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackParameters.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackParameters.cpp [ 8%] Building C object CMakeFiles/strumpack.dir/src/sparse/strumpack_mc64ad.c.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolver.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolver.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/StrumpackSparseSolverBase.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/StrumpackSparseSolverBase.cpp /usr/bin/mpicc -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -o CMakeFiles/strumpack.dir/src/sparse/strumpack_mc64ad.c.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/strumpack_mc64ad.c [ 8%] Building CXX object CMakeFiles/strumpack.dir/src/misc/TaskTimer.cpp.o [ 9%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRGraph.cpp.o [ 9%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/misc/TaskTimer.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/misc/TaskTimer.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRGraph.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRGraph.cpp /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRMatrix.cpp [ 10%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTree.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTree.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTree.cpp [ 10%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/SeparatorTree.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/SeparatorTree.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/SeparatorTree.cpp [ 12%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/CSRMatrixMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/CSRMatrixMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/CSRMatrixMPI.cpp [ 12%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTreeMPI.cpp [ 13%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPIDist.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/EliminationTreeMPIDist.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/EliminationTreeMPIDist.cpp [ 13%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/PropMapSparseMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/PropMapSparseMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/PropMapSparseMatrix.cpp [ 14%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStab.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStab.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/BiCGStab.cpp [ 16%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/GMRes.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/GMRes.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/GMRes.cpp [ 16%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinement.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinement.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/IterativeRefinement.cpp [ 17%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/GMResMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/GMResMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/GMResMPI.cpp [ 17%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStabMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/BiCGStabMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/BiCGStabMPI.cpp [ 18%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinementMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/iterative/IterativeRefinementMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/iterative/IterativeRefinementMPI.cpp [ 18%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontFactory.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontFactory.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontFactory.cpp [ 20%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrix.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrix.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrix.cpp [ 21%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDense.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDense.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixDense.cpp [ 21%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSS.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSS.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixHSS.cpp [ 22%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixMPI.cpp [ 22%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDenseMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixDenseMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixDenseMPI.cpp [ 24%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSSMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixHSSMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixHSSMPI.cpp [ 24%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixBLRMPI.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixBLRMPI.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixBLRMPI.cpp [ 25%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/ExtendAdd.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/ExtendAdd.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/ExtendAdd.cpp [ 25%] Building CXX object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixGPU.cpp.o /usr/bin/mpicxx -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem /home/nimda/sources/petsc/real_strumpack_cuda/include -isystem /usr/local/cuda-11.1/include -fstack-protector -O3 -march=native -fstack-protector -O3 -march=native -Wall -Wno-overloaded-virtual -Wno-unknown-pragmas -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixGPU.cpp.o -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixGPU.cpp [ 26%] Building CUDA object CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixCUDA.cu.o /usr/local/cuda-11.1/bin/nvcc -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src -I/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build -isystem=/home/nimda/sources/petsc/real_strumpack_cuda/include -O3 -DNDEBUG -std=c++11 -x cu -c /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixCUDA.cu -o CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixCUDA.cu.o make[2]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build' make[1]: Leaving directory '/home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/petsc-build'In file included from /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixGPUKernels.hpp:32, from /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/sparse/fronts/FrontalMatrixCUDA.cu:29: /home/nimda/sources/petsc/real_strumpack_cuda/externalpackages/git.strumpack/src/misc/Triplet.hpp:35:10: fatal error: mpi.h: No such file or directory 35 | #include | ^~~ compilation terminated. make[2]: [CMakeFiles/strumpack.dir/build.make:508: CMakeFiles/strumpack.dir/src/sparse/fronts/FrontalMatrixCUDA.cu.o] Error 1 make[2]: Waiting for unfinished jobs.... make[1]: [CMakeFiles/Makefile2:1028: CMakeFiles/strumpack.dir/all] Error 2 make: [Makefile:144: all] Error 2

pghysels commented 4 years ago

I see that when you run CMake, you specify -DCMAKE_CXX_COMPILER="mpicxx" so CMake detects that the CXX compiler is an MPI wrapper and will not look for the MPI headers and libraries, and thus things like MPI_CXX_COMPILE_DEFINITIONS and MPI_CXX_INCLUDE_PATH are empty, and nothing gets passed to nvcc. The issue is also discussed here: https://gitlab.kitware.com/cmake/cmake/-/issues/17538

If you can just not specify CMAKE_CXX_COMPILER at all, or have it point to a regular c++ compiler, then CMake should find the MPI headers and libraries and pass them to nvcc.

I will also try to find a different workaround. I might be able to separate the MPI code from the CUDA code.

pghysels commented 4 years ago

Perhaps you can also add: -DCMAKE_CUDA_HOST_COMPILER=which mpic++

pghysels commented 4 years ago

Hi, does the above suggestion work for you?

I haven't had time to modify the code. I would actually prefer not to split the MPI and CUDA code, as it would add more complexity to the code.

nindanaoto commented 4 years ago

Sorry for the delay. As far as I tested, this seems to not work. Did this workaround work in your environment?

nindanaoto commented 4 years ago

Just adding -DCMAKE_CUDA_FLAGS="-I/usr/lib/x86_64-linux-gnu/openmpi/include" seems to be work. (I did not know how to pass compile flags to nvcc......) I think this or something like this should be added to CMakeLists.txt.