slepc/step-36_parallel_02 failing

[masterleinad]

Currently, I get

8360: DEAL::Jacobi 8360: DEAL::KrylovSchur 8360: 8360: slepc/step-36_parallel_02.mpirun=3.debug: RUN failed. ------ Additional output on stdout/stderr: 8360: 8360: [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- 8360: [0]PETSC ERROR: No support for this operation for this object type 8360: [0]PETSC ERROR: You chose to solve linear systems with a factorization, but in parallel runs you need to select an external package; see the users guide for details 8360: [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. 8360: [0]PETSC ERROR: Petsc Release Version 3.6.3, Dec, 03, 2015 8360: [0]PETSC ERROR: /mnt/data/darndt/Sources/dealii/build/tests/slepc/step-36_parallel_02.debug/step-36_parallel_02.debug on a arch-linux2-c-opt named simserv02 by darndt Sun Sep 24 14:48:35 2017 8360: [0]PETSC ERROR: Configure options --with-debugging=0 --download-hypre --with-64-bit-indices

This seems to be similar to #1117.

[drwells]

I can confirm that this is essentially the same problem: I can run the test manually as

$ cd tests/slepc/step-36_parallel_02.debug/
$ mpirun -np 3 ./step-36_parallel_02.debug -st_pc_type bjacobi

which is the suggestion given in the SLEPc users manual (section 3.4, page 46). The test fails in this mode (the eigenvectors are not sufficiently orthogonal; we should fix that independently by asking SLEPc to reorthogonalize). The manual does specify that the default solver for the KSP solver is an LU solver, which is exactly our issue. My version of PETSc provides additional output:

$ ./step-36_parallel_02.debug
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/linearsolvertable.html for possible LU and Cholesky solvers
[0]PETSC ERROR: Could not locate a solver package. Perhaps you must ./configure with --download-<package>
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.7.6, Apr, 24, 2017
[0]PETSC ERROR: ./step-36_parallel_02.debug on a arch-linux-cxx-opt named archway by drwells Mon Sep 25 21:45:58 2017
[0]PETSC ERROR: Configure options --prefix=/opt/petsc/linux-cxx-opt
--PETSC_ARCH=arch-linux-cxx-opt --with-shared-libraries=1 --download-hypre=1
--with-64-bit-indices --with-suitesparse=0 --with-metis=0 --with-clanguage=C++
--COPTFLAGS=-O3 --CXXOPTFLAGS=-O3
[0]PETSC ERROR: #1 MatGetFactor() line 4244 in [...]/petsc/src/petsc-3.7.6/src/mat/interface/matrix.c
[0]PETSC ERROR: #2 PCSetUp_LU() line 134 in [...]/petsc/src/petsc-3.7.6/src/ksp/pc/impls/factor/lu/lu.c
[0]PETSC ERROR: #3 PCSetUp() line 968 in [...]/petsc/src/petsc-3.7.6/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: #4 KSPSetUp() line 390 in [...]/petsc/src/petsc-3.7.6/src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: #5 STSetUp_Shift() line 140 in [...]/slepc/src/slepc-3.7.4/src/sys/classes/st/impls/shift/shift.c
[0]PETSC ERROR: #6 STSetUp() line 301 in [...]/slepc/src/slepc-3.7.4/src/sys/classes/st/interface/stsolve.c
[0]PETSC ERROR: #7 EPSSetUp() line 205 in [...]/slepc/src/slepc-3.7.4/src/eps/interface/epssetup.c
[0]PETSC ERROR: #8 EPSSolve() line 89 in [...]/slepc/src/slepc-3.7.4/src/eps/interface/epssolve.c
ERROR: Uncaught exception in MPI_InitFinalize on proc 0. Skipping MPI_Finalize() to avoid a deadlock.

[davydden]

I think the problem is that your PETSc has neither SuperLU nor MUMPS. We need to switch this test to use iterative solvers for spectral transformation

[davydden]

a better approach would be to adopt https://github.com/dealii/dealii/pull/4994

[drwells]

I have only used SLEPc for SVD calculations and I have not been impacted by this problem in my own work. @davydden You are the SLEPc expert here; would it be particularly difficult to add an option to what we currently have to set up the KSP object with a non-LU solver?

[davydden]

KSP object with a non-LU solver

you can already do that https://github.com/dealii/dealii/blob/master/tests/slepc/step-36_parallel.cc#L286-L288

[davydden]

i think the tests should be just renamed to have petsc_with_mumps.