AMReX-Codes / amrex

AMReX: Software Framework for Block Structured AMR
https://amrex-codes.github.io/amrex
Other
503 stars 336 forks source link

add AMG-DD support to MLHypre #3998

Open BenWibking opened 1 week ago

BenWibking commented 1 week ago

Summary

This adds experimental support for the AMG-DD solver to the multi-level Hypre solver.

Note that for this solver, the residual is not independent of the domain decomposition.

Additional background

With 1 ghost layer, the MPI message rate and bandwidth is greatly reduced compared to BoomerAMG: https://arxiv.org/abs/1906.10575

Checklist

The proposed changes:

BenWibking commented 1 week ago

This currently crashes to due a possible Hypre bug:

[./../parcsr_mv/_hypre_parcsr_mv.h, 434] hypre_assert failed: 0
Assertion failed: (0), function hypre_ParCSRMatrixMemoryLocation, file _hypre_parcsr_mv.h, line 434.

Backtrace:

 0: amrex::BLBackTrace::print_backtrace_info(__sFILE*) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_BLBackTrace.cpp:190)

 1: amrex::BLBackTrace::handler(int) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_BLBackTrace.cpp:98)

 2: _sigtramp (in libsystem_platform.dylib) + 56

 3: pthread_kill (in libsystem_pthread.dylib) + 288

 4: abort (in libsystem_c.dylib) + 180

 5: err (in libsystem_c.dylib) + 0

 6: hypre_ParCSRMatrixMemoryLocation (in main3d.gnu.DEBUG.MPI.ex) + 464

 7: hypre_BoomerAMGSetup (in main3d.gnu.DEBUG.MPI.ex) + 220

 8: hypre_BoomerAMGDDSetup (in main3d.gnu.DEBUG.MPI.ex) (par_amgdd_setup.c:77)

 9: HYPRE_BoomerAMGDDSetup (in main3d.gnu.DEBUG.MPI.ex) (HYPRE_parcsr_amgdd.c:47)

10: amrex::HypreMLABecLap::solve(amrex::Vector<amrex::MultiFab*, std::__1::allocator<amrex::MultiFab*>> const&, amrex::Vector<amrex::MultiFab const*, std::__1::allocator<amrex::MultiFab const*>> const&, double, double) (in main3d.gnu.DEBUG.MPI.ex) (AMReX_HypreMLABecLap.cpp:1011)

11: MyTest::solveMLHypre() (in main3d.gnu.DEBUG.MPI.ex) (MyTest.cpp:670)

12: MyTest::solve() (in main3d.gnu.DEBUG.MPI.ex) (MyTest.cpp:29)

13: main (in main3d.gnu.DEBUG.MPI.ex) (main.cpp:12)