search

ECP-WarpX / WarpX

WarpX is an advanced electromagnetic & electrostatic Particle-In-Cell code.
https://ecp-warpx.github.io
Other
307 stars 195 forks source link

`nan` resulting from `meanParticleVelocity` during Poisson solve #4230

Open aeriforme opened 1 year ago

aeriforme commented 1 year ago

Hi all. I'm running a 3D simulation that, depending on the resolution, gives some nan values in some of the reduced diagnostics (e.g. ParticleEnergy and BeamRelevant) only at the last timestep. If I reduce the simulation time, I still get the nan values at the last timestep, even if the corresponding value was not nan in the longer run.

If I add to the the input these flags 

fab.init_snan = 1
amrex.fpe_trap_invalid = 1
amrex.fpe_trap_zero = 1
amrex.fpe_trap_overflow = 1

then the simulation crashes because of Erroneous arithmetic operation.

I'm using the relativistic electrostatic solver.

Using the Backtraces, the issues seems to arise during the Poisson solve. After adding some prints here and there, I understand that the average velocity of at least one species is nan.

Specifically, the meanParticleVelocity function defined in WarpXParticleContainer.cpp gives vx_total, vy_total, vz_total equal to nan, as computed here.

Note that the particles have not left the box at the considered timestep.

I'm running on 8 Perlmutter GPU nodes.

RemiLehe commented 7 months ago

Would you be able to check whether this issue is fixed in the current version of WarpX? (e.g. because of this PR: https://github.com/ECP-WarpX/WarpX/pull/4614)

aeriforme commented 7 months ago

I see the same behavior I reported with ef6d0853b0115dbbaf95b74d87c3367a4104b5cd.