Multiple GPUs RZ instability w/ Boundary "None"

[juliettepech]

I ran a RZ simulation with one GPU and get this first animation : Then, I used 4 GPUs instead of one. The only changes in the input files are the following : Using those conditions, an instability occurs:

The instability remains for different tests :

• [x] Higher resolution
• [x] Higher number of guard cells (psatd.nz_guard = 64)
• [x] No pml for boundary conditions
• [x] New version of WarpX : git describe gives now 23.08-11-g5313c5d93
• [x] New version of WarpX and psatd.current_correction = 0

The input file is the following: inputs_HTT_severalGPUs(instability).txt

[RemiLehe]

Thanks a lot for reporting this @juliettepech! Would you have time to check a few more things:

• [x] Does this also happen when running on CPU only?
• [x] Does this also happen for version 23.06 of WarpX?
• [x] Is the growth rate (e.g. as visualized by plotting the amplitude of Ez in log scale) different between psatd.nz_guard = 32 and psatd.nz_guard = 64?

[juliettepech]

I have finished running the last tests and here are the results:

[ax3l]

let's try:

• [ ] warpx.safe_guard_cells = 1
• [ ] algo.maxwell_solver = yee
• [ ] WarpX 23.01

let's debug via:

• [ ] https://warpx.readthedocs.io/en/latest/usage/workflows/debugging.html with only a few 1000 steps until the issue becomes visible
  • maybe it crashes earlier and we see where

[RemiLehe]

Let us also try without ionization (i.e. with pre-ionized plasma).

[juliettepech]

A few updates on the latest cases: Using WarpX 23.01 did not changed anything, the instability is still here. Same thing using warpx.safe_guard_cells = 1 and trying without ionization.

However, when the Yee Maxwell solver is used, the instability disappears. I also tried Jean-Luc's idea of experimenting with damping (damped boundary conditions for the z axis). Here again, the instability does not occur (the simulation is much longer though, it has reached the time limit of 5 hours, I can run it until the end if necessary).

[EZoni]

@juliettepech We actually decided to stop supporting none as boundary condition with PSATD, except for r=0 in RZ geometry; damped should be used instead. See PR #4210.