search

ECP-WarpX / WarpX

WarpX is an advanced electromagnetic & electrostatic Particle-In-Cell code.
https://ecp-warpx.github.io
Other
301 stars 191 forks source link

out-of-bounds when subcycling + back-transformed diagnostics #380

Open MaxThevenet opened 5 years ago

MaxThevenet commented 5 years ago

Compiling on dev branch with

make -j 8 DIM=2 DEBUG=TRUE

and running this inputs file results in 

MPI initialized with 1 MPI processes
OMP initialized with 1 OMP threads
AMReX (19.09-165-gcc850f3c226b-dirty) initialized

Grids Summary:
  Level 0   1 grids  32768 cells  100 % of domain
  Level 1   1 grids  7168 cells  5.46875 % of domain

  Writing plotfile diags/plotfiles/plt00000
max_step computed in computeMaxStepBoostAccelerator: 159

STEP 1 starts ...
amrex::Abort::0:: (122,210,0,0) is out of bound (111:145,199:209,0:0,0:0) !!!
SIGABRT
Failed to start process notifications for pid 34300 (19)
Failed to start process notifications for pid 34300 (19)
Failed to start process notifications for pid 34300 (19)
Failed to start process notifications for pid 34300 (19)
See Backtrace.0.0 file for details

With Backtrace

 0: amrex::BLBackTrace::print_backtrace_info(__sFILE*) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX_BLBackTrace.cpp:163)

 1: amrex::BLBackTrace::handler(int) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX_BLBackTrace.cpp:71)

 2: amrex::ParallelDescriptor::Abort(int, bool) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX_ParallelDescriptor.cpp:224)

 3: amrex::Abort(char const*) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX.cpp:198)

 4: amrex::Abort(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX.cpp:209)

 5: amrex::Array4<double>::index_assert(int, int, int, int) const (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX_Array4.H:181)

 6: void doChargeDepositionShapeN<1>(double const*, double const*, double const*, double const*, int const*, amrex::Array4<double> const&, long, std::array<double, 3ul> const&, std::array<double, 3ul>, amrex::Dim3, double)::'lambda'(long)::operator()(long) const (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (:-396370672)

 7: void amrex::ParallelFor<long, void doChargeDepositionShapeN<1>(double const*, double const*, double const*, double const*, int const*, amrex::Array4<double> const&, long, std::array<double, 3ul> const&, std::array<double, 3ul>, amrex::Dim3, double)::'lambda'(long), void>(long, void doChargeDepositionShapeN<1>(double const*, double const*, double const*, double const*, int const*, amrex::Array4<double> const&, long, std::array<double, 3ul> const&, std::array<double, 3ul>, amrex::Dim3, double)::'lambda'(long)&&, unsigned long) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (AMReX_GpuLaunchFunctsC.H:24)

 8: void doChargeDepositionShapeN<1>(double const*, double const*, double const*, double const*, int const*, amrex::Array4<double> const&, long, std::array<double, 3ul> const&, std::array<double, 3ul>, amrex::Dim3, double) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (ChargeDeposition.H:106)

 9: WarpXParticleContainer::DepositCharge(WarpXParIter&, amrex::Vector<double, std::allocator<double> >&, int const*, amrex::MultiFab*, int, long, long, int, int, int) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (WarpXParticleContainer.cpp:518)

10: WarpXParticleContainer::GetChargeDensity(int, bool) (._omp_fn.21) (in main2d.gnu.DEBUG.TPROF.MPI.OMP.ex) (WarpXParticleContainer.cpp:642)

11: GOMP_parallel (in libgomp.1.dylib) + 63

===== TinyProfilers ======
main()
REG::WarpX::Evolve()
WarpX::EvolveEM()
WarpX::GetCellCenteredData
PPC::ChargeDeposition

It seems to be independent from PR https://github.com/ECP-WarpX/WarpX/pull/377

ax3l commented 5 years ago

Charge deposition was last changed in #263, maybe @dpgrote can spot if there is an out-of-bounds access in it?

MaxThevenet commented 5 years ago

Note that this happens when using back-transformed because rho is requested by default in back-transformed diagnostics, so we need to do the charge deposition after the full timestep.

RemiLehe commented 4 years ago

Could this be related to #293 ?

RemiLehe commented 4 years ago

and to #22 ?