Open BenWibking opened 1 week ago
This currently crashes to due a possible Hypre bug:
[./../parcsr_mv/_hypre_parcsr_mv.h, 434] hypre_assert failed: 0
Assertion failed: (0), function hypre_ParCSRMatrixMemoryLocation, file _hypre_parcsr_mv.h, line 434.
Backtrace:
0: amrex::BLBackTrace::print_backtrace_info(__sFILE*) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_BLBackTrace.cpp:190)
1: amrex::BLBackTrace::handler(int) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_BLBackTrace.cpp:98)
2: _sigtramp (in libsystem_platform.dylib) + 56
3: pthread_kill (in libsystem_pthread.dylib) + 288
4: abort (in libsystem_c.dylib) + 180
5: err (in libsystem_c.dylib) + 0
6: hypre_ParCSRMatrixMemoryLocation (in main3d.gnu.DEBUG.MPI.ex) + 464
7: hypre_BoomerAMGSetup (in main3d.gnu.DEBUG.MPI.ex) + 220
8: hypre_BoomerAMGDDSetup (in main3d.gnu.DEBUG.MPI.ex) (par_amgdd_setup.c:77)
9: HYPRE_BoomerAMGDDSetup (in main3d.gnu.DEBUG.MPI.ex) (HYPRE_parcsr_amgdd.c:47)
10: amrex::HypreMLABecLap::solve(amrex::Vector<amrex::MultiFab*, std::__1::allocator<amrex::MultiFab*>> const&, amrex::Vector<amrex::MultiFab const*, std::__1::allocator<amrex::MultiFab const*>> const&, double, double) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_HypreMLABecLap.cpp:1011)
11: MyTest::solveMLHypre() (in main3d.gnu.DEBUG.MPI.ex) (MyTest.cpp:670)
12: MyTest::solve() (in main3d.gnu.DEBUG.MPI.ex) (MyTest.cpp:29)
13: main (in main3d.gnu.DEBUG.MPI.ex) (main.cpp:12)
Summary
This adds experimental support for the AMG-DD solver to the multi-level Hypre solver.
Note that for this solver, the residual is not independent of the domain decomposition.
Additional background
With 1 ghost layer, the MPI message rate and bandwidth is greatly reduced compared to BoomerAMG: https://arxiv.org/abs/1906.10575
Checklist
The proposed changes: