search

boutproject / BOUT-dev

BOUT++: Plasma fluid finite-difference simulation code in curvilinear coordinate systems
http://boutproject.github.io/
GNU Lesser General Public License v3.0
182 stars 95 forks source link

next test failures #1981

Closed dschwoerer closed 3 years ago

dschwoerer commented 4 years ago

One example fails to compile:

Trying to compile ../../../examples/laplacexy/alfven-wave
  Compiling  alfven.cxx
In file included from alfven.cxx:2:
../../../include/bout/invert/laplacexy.hxx:39:2: warning: #warning LaplaceXY requires PETSc. No LaplaceXY available [-Wcpp]
   39 | #warning LaplaceXY requires PETSc. No LaplaceXY available
      |  ^~~~~~~
alfven.cxx: In member function ‘virtual int Alfven::init(bool)’:
alfven.cxx:81:33: error: ‘mesh’ was not declared in this scope; did you mean ‘bout::globals::mesh’?
   81 |       laplacexy = new LaplaceXY(mesh);
      |                                 ^~~~
      |                                 bout::globals::mesh
In file included from ../../../include/field_data.hxx:46,
                 from ../../../include/field2d.hxx:35,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
../../../include/boundary_region.hxx:13:16: note: ‘bout::globals::mesh’ declared here
   13 |   extern Mesh* mesh; ///< Global mesh
      |                ^~~~
alfven.cxx:89:37: error: ‘mesh’ was not declared in this scope; did you mean ‘bout::globals::mesh’?
   89 |       newSolver = LaplaceXZ::create(mesh);
      |                                     ^~~~
      |                                     bout::globals::mesh
In file included from ../../../include/field_data.hxx:46,
                 from ../../../include/field2d.hxx:35,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
../../../include/boundary_region.hxx:13:16: note: ‘bout::globals::mesh’ declared here
   13 |   extern Mesh* mesh; ///< Global mesh
      |                ^~~~
alfven.cxx: In member function ‘virtual int Alfven::rhs(BoutReal)’:
alfven.cxx:108:5: error: ‘mesh’ was not declared in this scope; did you mean ‘bout::globals::mesh’?
  108 |     mesh->communicate(Vort, Apar);
      |     ^~~~
      |     bout::globals::mesh
In file included from ../../../include/field_data.hxx:46,
                 from ../../../include/field2d.hxx:35,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
../../../include/boundary_region.hxx:13:16: note: ‘bout::globals::mesh’ declared here
   13 |   extern Mesh* mesh; ///< Global mesh
      |                ^~~~
In file included from ../../../include/field.hxx:40,
                 from ../../../include/field2d.hxx:34,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
alfven.cxx: In member function ‘void Alfven::LoadMetric(BoutReal, BoutReal)’:
../../../include/globals.hxx:64:5: error: ‘mesh’ was not declared in this scope; did you mean ‘bout::globals::mesh’?
   64 |     mesh->get(var1, #var1); \
      |     ^~~~
alfven.cxx:172:5: note: in expansion of macro ‘GRID_LOAD5’
  172 |     GRID_LOAD5(Rxy, Bpxy, Btxy, hthe, sinty); // Load metrics
      |     ^~~~~~~~~~
In file included from ../../../include/field_data.hxx:46,
                 from ../../../include/field2d.hxx:35,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
../../../include/boundary_region.hxx:13:16: note: ‘bout::globals::mesh’ declared here
   13 |   extern Mesh* mesh; ///< Global mesh
      |                ^~~~
alfven.cxx:174:26: error: ‘mesh’ was not declared in this scope; did you mean ‘bout::globals::mesh’?
  174 |     Coordinates *coord = mesh->getCoordinates(); // Metric tensor
      |                          ^~~~
      |                          bout::globals::mesh
In file included from ../../../include/field_data.hxx:46,
                 from ../../../include/field2d.hxx:35,
                 from ../../../include/options.hxx:49,
                 from ../../../include/bout/deriv_store.hxx:42,
                 from ../../../include/bout/mesh.hxx:49,
                 from ../../../include/bout/invert/laplacexy.hxx:41,
                 from alfven.cxx:2:
../../../include/boundary_region.hxx:13:16: note: ‘bout::globals::mesh’ declared here
   13 |   extern Mesh* mesh; ///< Global mesh
      |                ^~~~
make: *** [../../../make.config:318: alfven.o] Error 1
alfven-wave failed

test-communications ❌ 6.762 s I have fix, need to cherry-pick

test-laplace-petsc3d failes, output not useful (should be fixed as well)

----- test-laplace-petsc3d -----
Making Laplace 3D with PETSc
running on 1 processors: test-laplace3d -d data_slab_core > data_slab_core/output.log 2>&1
getmpirun: using the default mpirun -np
data_slab_core run crashed
running on 1 processors: test-laplace3d -d data_slab_sol > data_slab_sol/output.log 2>&1
getmpirun: using the default mpirun -np
data_slab_sol run crashed
running on 1 processors: test-laplace3d -d data_circular_core > data_circular_core/output.log 2>&1
getmpirun: using the default mpirun -np
data_circular_core run crashed
running on 1 processors: test-laplace3d -d data_circular_core-sol > data_circular_core-sol/output.log 2>&1
getmpirun: using the default mpirun -np
data_circular_core-sol run crashed
dschwoerer commented 4 years ago

test-laplace-petsc3d still fails.

    Option input:transform_from_field_aligned = false (data_slab_core/BOUT.inp)
    Option input:max_recursion_depth = 0 (default)
    Option d:function = 1. + .1*mixmode(x, 7.)*mixmode(y, 8.)*mixmode(z, 9.) (data_slab_core/BOUT.inp)
    Option all:scale = 1 (default)
[0]PETSC ERROR: --------------------- Error Message ------------------------------------
[0]PETSC ERROR: Arguments are incompatible!
[0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, seqaij!
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Petsc Release Version 3.4.5, Jun, 29, 2014 
[0]PETSC ERROR: See docs/changes/index.html for recent updates.
[0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
[0]PETSC ERROR: See docs/index.html for manual pages.
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: ./test-laplace3d on a arch-linux2-cxx-debug named omadesala.alfa1024.info by bout-test Wed Mar 25 02:33:48 2020
[0]PETSC ERROR: Libraries linked from /home/bout-test/petsc/petsc-3.4.5/arch-linux2-cxx-debug/lib
[0]PETSC ERROR: Configure run at Wed Jan 16 14:12:53 2019
[0]PETSC ERROR: Configure options --download-f-blas-lapack=1 --with-clanguage=cxx --with-mpi=yes --with-precision=double --with-scalar-type=real
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: MatMatMult() line 8649 in src/mat/interface/matrix.c
[0]PETSC ERROR: PCGAMGOptprol_AGG() line 1327 in src/ksp/pc/impls/gamg/agg.c
[0]PETSC ERROR: PCSetUp_GAMG() line 682 in src/ksp/pc/impls/gamg/gamg.c
[0]PETSC ERROR: PCSetUp() line 890 in src/ksp/pc/interface/precon.c
[0]PETSC ERROR: KSPSetUp() line 278 in src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: KSPSolve() line 399 in src/ksp/ksp/interface/itfunc.c
KSPConvergedReason is 0
terminate called after throwing an instance of 'BoutException'
  what():  Petsc3dAmg: inversion failed to converge.

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 423 RUNNING AT omadesala.alfa1024.info
=   EXIT CODE: 134
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Aborted (signal 6)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
Making Laplace 3D with PETSc
running on 1 processors: ./test-laplace3d -d data_slab_core
getmpirun: using the default mpirun -np
Traceback (most recent call last):
  File "./runtest", line 22, in 
    launch_safe(command, nproc=nproc)
  File "/home/bout-test/tmp.9jCljI5AiC/tools/pylib/boututils/run_wrapper.py", line 289, in launch_safe
    (s,command,out))
    Option input:transform_from_field_aligned = false (data_slab_core/BOUT.inp)
    Option input:max_recursion_depth = 0 (default)
    Option d:function = 1. + .1*mixmode(x, 7.)*mixmode(y, 8.)*mixmode(z, 9.) (data_slab_core/BOUT.inp)
    Option all:scale = 1 (default)
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Arguments are incompatible
[0]PETSC ERROR: MatMatMult requires A, mpiaij, to be compatible with B, seqaij
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.5.4, May, 23, 2015 
[0]PETSC ERROR: ./test-laplace3d on a arch-linux2-cxx-debug named omadesala.alfa1024.info by bout-test Wed Mar 25 03:00:08 2020
[0]PETSC ERROR: Configure options --download-fblaslapack=1 --with-clanguage=cxx --with-mpi=yes --with-precision=double --with-scalar-type=real
[0]PETSC ERROR: #1 MatMatMult() line 8710 in /home/bout-test/petsc/petsc-3.5.4/src/mat/interface/matrix.c
[0]PETSC ERROR: #2 PCGAMGOptprol_AGG() line 1328 in /home/bout-test/petsc/petsc-3.5.4/src/ksp/pc/impls/gamg/agg.c
[0]PETSC ERROR: #3 PCSetUp_GAMG() line 606 in /home/bout-test/petsc/petsc-3.5.4/src/ksp/pc/impls/gamg/gamg.c
[0]PETSC ERROR: #4 PCSetUp() line 902 in /home/bout-test/petsc/petsc-3.5.4/src/ksp/pc/interface/precon.c
[0]PETSC ERROR: #5 KSPSetUp() line 306 in /home/bout-test/petsc/petsc-3.5.4/src/ksp/ksp/interface/itfunc.c
[0]PETSC ERROR: #6 KSPSolve() line 418 in /home/bout-test/petsc/petsc-3.5.4/src/ksp/ksp/interface/itfunc.c
KSPConvergedReason is 0
terminate called after throwing an instance of 'BoutException'
  what():  Petsc3dAmg: inversion failed to converge.

PETSc 3.6 got stuck

PETSc 3.7, 3.8 and 3.9 passes

johnomotani commented 4 years ago

@ZedThree do you remember if Petsc3dAmg is supposed to work with Petsc-3.4? I can't find the discussion or a check in the code now, but I have a feeling that at some point we required 3.6 or even 3.8?

ZedThree commented 4 years ago

Yeah, I'm reasonably certain we decided 3.4 was not important to support. Release date was 2013.

It would definitely be nice if it worked with 3.6, as that's what's on Ubuntu 16, and I figure we're still trying to support that till EOL next year?

We should maybe bump up required PETSc version number to at least 3.6 in both CMake and configure.

ZedThree commented 4 years ago

Found some relevant comments. There was a problem with PETSc < 3.8, but it looks like we fixed it:

https://github.com/boutproject/BOUT-dev/pull/1803#issuecomment-583518658 https://github.com/boutproject/BOUT-dev/pull/1803#issuecomment-585731351 https://github.com/boutproject/BOUT-dev/pull/1803/commits/32c7d40f798ae24d57893f4373570d83c80fe874

So I did get it to work with PETSc 3.6 at least.

dschwoerer commented 4 years ago

So it should be working with petsc 3.6, but not with 3.5? Would be nice if we could say that a test #requires petsc >= 3.6

However, it doesn't work for me with 3.6 ... This is on fedora 30, most recent petsc 3.6, compiled with gcc and mpich-x86_64

ZedThree commented 3 years ago

I think this was fixed in #2042. Ubuntu 16.04 is EOL at the end of this month, so I'm not sure there's still reason to support PETSc 3.6