search

ECP-WarpX / WarpX

WarpX is an advanced electromagnetic & electrostatic Particle-In-Cell code.
https://ecp-warpx.github.io
Other
292 stars 188 forks source link

Too much memory required for checking domain with moving window #5149

Open Poto45 opened 1 month ago

Poto45 commented 1 month ago

Hello! I've been playing around with having z-dependent epsilon and sigma functions with a moving window (#4952 and, consequently, #5082), but have been having issues with memory. I’ve attached the warpx input script I use and the run script I’ve been using for the cluster I’m on. For a simple gaussian case like this and for the domain of the moving window, I’m able to have n_cell go up to 256 256 2048. However, if I’m using an external file, the max number of cells I can have before it breaks is 32 32 256. It will break at 654 32 256 or anything with more number of cells.

There are two errors that I get. The more common one happens around starting step 2 (usually happens for the external file):

_amrex::Abort::1::CUDA error 700 in file
/home/piot/aard/philippe/codes/WarpX/build_24.07_macro/_deps/fetchedamrex-src/Src/Base/AMReX_GpuDevice.cpp line 648:
an illegal memory access was encountered !!!
SIGABRT_
The second one is while Level 0 is happening (usually happens with the gaussian created by WarpX): 
_amrex::Abort::0::Out of gpu memory. Free: 4456448 Asked: 8388608 !!!
SIGABRT_

Also attached are a sample log and a sample Backtrace file for both the gaussian and the external file cases. Please let me know if there’s anything I can try or any information I can get from the system. Thanks!

run_file.txt sqr_struct_external_backtrace.txt sqr_struct_external_log.txt sqr_struct_gauss_backtrace.txt sqr_struct_gauss_log.txt WX_inputfile.txt

RemiLehe commented 1 month ago

Thanks for reporting this issue. Could you post the input script that you use when injecting particles from an external file? Also: how many macroparticles does the external file contain? Is it much larger than the number of macroparticles for the gaussian beam created by WarpX (i.e. 30000 macroparticles)?

Poto45 commented 4 weeks ago

Hi Remi,

Attached is that file. The external file contains 30000 macroparticles, just like the gaussian beam. I've also sent you an email with the h5 particle file, since I can't attach it here. Thanks!

I did try running a simulation without including the z dependence in the epsilon and sigma functions and I'm able to get up to 256 256 1024, but it breaks above that. That is with the gaussian case.

ext_file_warpx.txt