loeiten commented 8 years ago

I'm trying building the next branch with PETSc, but it seems to be not working.

The summary of

./configure --with-checks=3 --with-track --with-debug --with-petsc --with-slepc --with-sundials

reads

Configuration summary

  FACETS support: no
  PETSc support: yes (version 3.4, release = 1)
  PETSc has SUNDIALS support: no
  SLEPc support: yes
  IDA support: yes
  CVODE support: yes
  ARKODE support: no
  NetCDF support: yes
  Parallel-NetCDF support: no
  HDF5 support: no (parallel: no)
  PDB support: no
  Hypre support: no
  MUMPS support: no

but

make

returns

----- Compiling BOUT++ -----
CXX      =  mpicxx
CFLAGS   =  -std=c++11 -O -DCHECK=2 -DSIGHANDLE -DBACKTRACE -DREVISION=1bb98bc74be926dd69d303a223e98642c3896f4f -DNCDF -DBOUT_HAS_PETSC -DBOUT_HAS_PETSC_3_4 -DBOUT_HAS_SLEPC -DBOUT_HAS_SLEPC_3_4 -DBOUT_HAS_IDA -DBOUT_HAS_CVODE -DCVODEINT=long -DIDAINT=long -DBOUT_HAS_PVODE
CHECKSUM =  3dea5bf1d71a26f2d03a914d91d9a6f7
INCLUDE  =  -I../include -I/home/mmag/petsc-3.4.5/include -I/home/mmag/petsc-3.4.5/arch-linux2-cxx-debug/include -I/home/mmag/anaconda3/include -I/home/mmag/slepc-3.4.4 -I/home/mmag/slepc-3.4.4/arch-linux2-cxx-debug/include -I/home/mmag/slepc-3.4.4/include -I/home/mmag/local/include -I/home/mmag/local/include
  Compiling  field.cxx
  Compiling  field2d.cxx
  Compiling  field3d.cxx
  Compiling  fieldperp.cxx
  Compiling  field_data.cxx
  Compiling  fieldgroup.cxx
  Compiling  field_factory.cxx
  Compiling  fieldgenerators.cxx
  Compiling  initialprofiles.cxx
  Compiling  vecops.cxx
  Compiling  vector2d.cxx
  Compiling  vector3d.cxx
  Compiling  where.cxx
  Compiling  globalfield.cxx
Adding field.o field2d.o field3d.o fieldperp.o field_data.o fieldgroup.o field_factory.o fieldgenerators.o initialprofiles.o vecops.o vector2d.o vector3d.o where.o globalfield.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  pdb_format.cxx
Adding pdb_format.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  nc_format.cxx
Adding nc_format.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  ncxx4.cxx
Adding ncxx4.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  pnetcdf.cxx
Adding pnetcdf.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  h5_format.cxx
Adding h5_format.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  datafile.cxx
  Compiling  dataformat.cxx
  Compiling  formatfactory.cxx
Adding datafile.o dataformat.o formatfactory.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  serial.cxx
Adding serial.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  cyclic.cxx
Adding cyclic.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  invert_parderiv.cxx
  Compiling  parderiv_factory.cxx
Adding invert_parderiv.o parderiv_factory.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  serial_tri.cxx
Adding serial_tri.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  serial_band.cxx
Adding serial_band.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  pdd.cxx
Adding pdd.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  spt.cxx
Adding spt.o to libbout++.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
  Compiling  petsc_laplace.cxx
In file included from petsc_laplace.hxx:59:0,
                 from petsc_laplace.cxx:28:
../../../../../include/bout/petsclib.hxx:51:0: warning: "PETSC_DEPRECATED" redefined
 #define PETSC_DEPRECATED(a)
 ^
In file included from /home/mmag/petsc-3.4.5/include/petscsys.h:14:0,
                 from /home/mmag/petsc-3.4.5/include/petscis.h:7,
                 from /home/mmag/petsc-3.4.5/include/petscvec.h:9,
                 from /home/mmag/petsc-3.4.5/include/petscmat.h:6,
                 from /home/mmag/petsc-3.4.5/include/petscpc.h:6,
                 from /home/mmag/petsc-3.4.5/include/petscksp.h:6,
                 from petsc_laplace.hxx:56,
                 from petsc_laplace.cxx:28:
/home/mmag/petsc-3.4.5/arch-linux2-cxx-debug/include/petscconf.h:25:0: note: this is the location of the previous definition
 #define PETSC_DEPRECATED(why) __attribute((deprecated))
 ^
petsc_laplace.cxx: In constructor ‘LaplacePetsc::LaplacePetsc(Options*)’:
petsc_laplace.cxx:94:25: warning: ‘MPI_Comm Mesh::getXcomm()’ is deprecated [-Wdeprecated-declarations]
   comm = mesh->getXcomm();
                         ^
In file included from ../../../../../include/field.hxx:39:0,
                 from ../../../../../include/field2d.hxx:34,
                 from ../../../../../include/bout/mesh.hxx:48,
                 from ../../../../../include/globals.hxx:30,
                 from petsc_laplace.hxx:54,
                 from petsc_laplace.cxx:28:
../../../../../include/bout/mesh.hxx:162:23: note: declared here
   DEPRECATED(MPI_Comm getXcomm()) {return getXcomm(0);}
                       ^
../../../../../include/bout/deprecated.hxx:9:26: note: in definition of macro ‘DEPRECATED’
 #define DEPRECATED(func) func __attribute__ ((deprecated))
                          ^
petsc_laplace.cxx: In member function ‘void LaplacePetsc::Coeffs(int, int, int, BoutReal&, BoutReal&, BoutReal&, BoutReal&, BoutReal&)’:
petsc_laplace.cxx:1008:83: error: call of overloaded ‘SQ(BoutReal&)’ is ambiguous
       coef4 -= 0.5 * ( ( coord->dx(x+1,y) - coord->dx(x-1,y) ) / SQ(coord->dx(x,y)) ) * coef1; // BOUT-06 term
                                                                                   ^
In file included from ../../../../../include/bout/mesh.hxx:48:0,
                 from ../../../../../include/globals.hxx:30,
                 from petsc_laplace.hxx:54,
                 from petsc_laplace.cxx:28:
../../../../../include/field2d.hxx:233:15: note: candidate: const Field2D SQ(const Field2D&)
 const Field2D SQ(const Field2D &f);
               ^
In file included from ../../../../../include/bout/mesh.hxx:49:0,
                 from ../../../../../include/globals.hxx:30,
                 from petsc_laplace.hxx:54,
                 from petsc_laplace.cxx:28:
../../../../../include/field3d.hxx:359:15: note: candidate: const Field3D SQ(const Field3D&)
 const Field3D SQ(const Field3D &f);
               ^
petsc_laplace.cxx:1012:12: error: ‘class Mesh’ has no member named ‘ShiftXderivs’
   if(mesh->ShiftXderivs && mesh->IncIntShear) {
            ^
../../../../../make.config:177: recipe for target 'petsc_laplace.o' failed
make[5]: *** [petsc_laplace.o] Error 1
../../../../make.config:86: recipe for target 'petsc' failed
make[4]: *** [petsc] Error 2
../../../make.config:86: recipe for target 'impls' failed
make[3]: *** [impls] Error 2
../../make.config:86: recipe for target 'laplace' failed
make[2]: *** [laplace] Error 2
../make.config:86: recipe for target 'invert' failed
make[1]: *** [invert] Error 2
make.config:86: recipe for target 'src' failed
make: *** [src] Error 2

However, this works on the master branch

ZedThree commented 8 years ago

Can you try the next-petsc-laplace-fixes I've just pushed? It compiles for me, but I don't have a suitable test case.

loeiten commented 8 years ago

Both building and running the tests works without any problems on the mentioned branch :)

ZedThree commented 8 years ago

Do you have a nice test case for petsc-laplace, by any chance?

loeiten commented 8 years ago

Answer: Yes...

...but not written by me. I tried to do a MES test of it a while back, but had problems with the solver throwing KSPConvergedReason -4 - "Laplacian inversion took too many iterations.". The test can probably be simplified and fixed for special cases of the mesh though. However, to just check that the petsc-laplace works, one can try

examples/blob2d/ - just change boussinesq = false in BOUT.inp

Side note

The test could also be working for examples/blob2d-laplacexz if one would like to try PETSc with external libraries for the preconditioner. With the current configuration, this throws

[0]PETSC ERROR: --------------------- Error Message ------------------------------------
[0]PETSC ERROR: No support for this operation for this object type!
[0]PETSC ERROR: Matrix format mpiaij does not have a built-in PETSc LU!
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Petsc Release Version 3.4.5, Jun, 29, 2014 
[0]PETSC ERROR: See docs/changes/index.html for recent updates.
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[0]PETSC ERROR: See docs/index.html for manual pages.
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: ./blob2d on a arch-linux2-cxx-debug named radegast by mmag Thu Nov  3 16:25:52 2016
[0]PETSC ERROR: Libraries linked from /home/mmag/petsc-3.4.5/arch-linux2-cxx-debug/lib
[0]PETSC ERROR: Configure run at Mon Aug 15 22:31:11 2016
[0]PETSC ERROR: Configure options --with-clanguage=cxx --with-mpi=1 --with-precision=double --with-scalar-type=real --with-shared-libraries=0 --download-fblaslapack=1 --download-f2cblaslapack=1
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: MatGetFactor() line 3966 in src/mat/interface/matrix.c
[0]PETSC ERROR: PCSetUp_LU() line 125 in src/ksp/pc/impls/factor/lu/lu.c
[0]PETSC ERROR: PCSetUp() line 890 in src/ksp/pc/interface/precon.c
[0]PETSC ERROR: KSPSetUp() line 278 in src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: KSPSolve() line 399 in src/ksp/ksp/interface/itfunc.c
Error encountered
====== Exception thrown ======
LaplaceXZ failed to converge. Reason 0
====== Back trace ======
 -> LaplaceXZpetsc::solve on line 572 of 'laplacexz-petsc.cxx'
====== Exception path ======
[bt] #1 ./blob2d() [0x46127f]
BoutException::BoutException(char const*, ...) at ??:?
[bt] #2 ./blob2d() [0x489daa]
LaplaceXZpetsc::solve(Field3D const&, Field3D const&) at ??:?
[bt] #3 ./blob2d() [0x409dd6]
physics_run(double) at ??:?
[bt] #4 ./blob2d() [0x45e0f2]
Solver::run_rhs(double) at ??:?
[bt] #5 ./blob2d() [0x45ea86]
Solver::solve(int, double) at ??:?
[bt] #6 ./blob2d() [0x46c938]
bout_run(Solver*, int (*)(double)) at ??:?
[bt] #7 ./blob2d() [0x40b5b5]
main at ??:?
[bt] #8 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f165c1d2830]
?? ??:0
[bt] #9 ./blob2d() [0x409a09]
_start at ??:?

or

[0]PETSC ERROR: Matrix format mpiaij does not have a solver package superlu_dist for LU. Perhaps you must ./configure with --download-superlu_dist!

if bousinesq = false in BOUT.inp.

The same goes for examples/test-laplacexz

ZedThree commented 7 years ago

This is (hopefully) fixed in #370.

ZedThree commented 7 years ago

Fixed in release candidate. Still needs a good test in examples though

boutproject / BOUT-dev

Building next with PETSc #356

Answer: Yes...

Side note