ECP-WarpX / WarpX

WarpX is an advanced electromagnetic & electrostatic Particle-In-Cell code.
https://ecp-warpx.github.io
Other
291 stars 185 forks source link

CFL limitation for electrostatic simulations #5113

Open archermarx opened 1 month ago

archermarx commented 1 month ago

Hi all,

Following on some issues encountered in this issue, I was wondering if it would be reasonable to develop an option for CFL-limited timesteps in electrostatic simulations. The basic idea would be to limit the timestep such that the fastest particle in the domain cannot cross more than one cell in a single timestep. Failing that, it would be good to at least warn the user if the chosen timestep and particle velocities lead to CFL condition violations.

The benefits of this would be twofold. First, it would prevent issues like this one where particles cannot be brought back into the domain with a single periodic shift, leading to segfaults. A more informative error would help significantly, but even better would be to prevent such an issue from arising in the first place. Second, it would potentially accelerate some simulations. In my work, I often have to use overly-pessimistic timesteps to avoid CFL condition violations in case a fast tail of electrons develops or unexpected heating occurs.

If this seems like a good idea, I can take a stab at implementing this early next week.

ax3l commented 1 month ago

Interesting that it does not hit this safe guard, https://github.com/ECP-WarpX/WarpX/blob/502e5ea31c1334f0083bed64eb8f8202b1100e81/Source/ablastr/particles/DepositCharge.H#L128-L130 which is supposed to give us a cleaner error message than what you encountered https://github.com/ECP-WarpX/WarpX/issues/5065#issue-2419886602

Maybe we need to add this at another location, too.

ax3l commented 1 month ago

The if (cur_time + dt[0] >= stop_time - 1.e-3*dt[0] || step == numsteps_max-1) { is the exit criteria, where we just do a half-step push to synchronize times for diagnsotics. You could try to do a Redistribute() at the beginning and end of this if block to make sure particles are in the right boxes?

ax3l commented 1 month ago

The call you might want to use is HandleParticlesAtBoundaries https://github.com/ECP-WarpX/WarpX/blob/502e5ea31c1334f0083bed64eb8f8202b1100e81/Source/Evolve/WarpXEvolve.cpp#L226

archermarx commented 1 month ago

Interesting that it does not hit this safe guard,

https://github.com/ECP-WarpX/WarpX/blob/502e5ea31c1334f0083bed64eb8f8202b1100e81/Source/ablastr/particles/DepositCharge.H#L128-L130

which is supposed to give us a cleaner error message than what you encountered #5065 (comment)

Maybe we need to add this at another location, too.

Interesting, I feel like I've gotten this error in the past (maybe 5 months ago?)

archermarx commented 1 month ago

The call you might want to use is HandleParticlesAtBoundaries

https://github.com/ECP-WarpX/WarpX/blob/502e5ea31c1334f0083bed64eb8f8202b1100e81/Source/Evolve/WarpXEvolve.cpp#L226

Thanks @ax3l. Throwing in a call to HandleParticlesAtBoundaries at the start of the conditional completely fixed the problem. I'll go ahead and make a PR