search

jzuhone / pyxsim

Simulating X-ray observations from astrophysical sources.
http://hea-www.cfa.harvard.edu/~jzuhone/pyxsim
Other
20 stars 8 forks source link

process_data in thermal_sources fails under certain conditions #49

Closed rjtanner closed 10 months ago

rjtanner commented 1 year ago

I am using version 4.1.1

Inside the for loop starting at line 357 in thermal_sources.py, my simulations are failing at lines 402-404 (I don't know which line). It happens when cn is large, > 10000000. When it fails, Python does not throw an exception or produce an error. The process is simply killed. It is possible it is a memory error because cn is so large the arrays generated should be massive.

Relevant lines where the code fails:

402 randvec = self.prng.uniform(size=cn) 403 randvec.sort() 404 cell_e = np.interp(randvec, cp[icell, :], self.bin_edges)

rjtanner commented 1 year ago

After some more testing, this is an issue with my data(-ish). In my data there are a few cells that just produce a huge number of photons (high density, high temperature, and/or high velocities). The excessive number of photons generated from a single cell causes memory issues because of the way prng.uniform and .sort() work. I applied a filter to my data to remove these extreme point sources and now it works.

jzuhone commented 1 year ago

@rjtanner can we close this or should we address it somehow?