Indexerror within make_photon function

[Rongjun-ANU]

Hi, I meet an index error when using make_photon function in pyxsim

I have loaded a yt object called ds_plt1555000; the shape is 1281281024 I used ad_plt1555000 = ds_plt1555000.all_data()

Here is the issue:

---

sp_plt1555000 = ds_plt1555000.sphere("c", (10, "kpc"))

Generate the photon list

n_photons, n_cells = pyxsim.make_photons("plt1555000_photon_list", data_source=sp_plt1555000, redshift=redshift, area=area, exp_time=exp_time, source_model=source_model)

pyxsim : [INFO ] 2023-09-26 12:19:14,484 Cosmology: h = 0.71, omega_matter = 0.27, omega_lambda = 0.73 pyxsim : [INFO ] 2023-09-26 12:19:14,486 Using emission measure field '('gas', 'emission_measure')'. pyxsim : [INFO ] 2023-09-26 12:19:14,486 Using temperature field '('gas', 'temperature')'. Preparing spectrum table : 100% 138/138 [00:37<00:00, 81.02it/s] Processing cells/particles : 100% 16777216/16777216 [03:33<00:00, 92569.44it/s]

---

IndexError Traceback (most recent call last) /home/rongjun/VSCAA/ASTR8010/try/Dash.ipynb Cell 59 line 1 7 region = ds_plt1555000.region(domain_center, domain_left_edge, domain_right_edge) 8 # ds_p = ds_plt1555000.force_periodicity() 9 10 # Generate the photon list ---> 11 n_photons, n_cells = pyxsim.make_photons("plt1555000_photon_list", 12 data_source=sp_plt1555000, 13 redshift=redshift, area=area, 14 exp_time=exp_time, source_model=source_model)

File ~/.local/lib/python3.8/site-packages/pyxsim/photon_list.py:416, in make_photons(photon_prefix, data_source, redshift, area, exp_time, source_model, point_sources, parameters, center, dist, cosmology, velocity_fields, bulk_velocity, observer, fields_to_keep) 413 d["energy"].resize((p_size,)) 415 for i, ax in enumerate("xyz"): --> 416 pos = chunk[p_fields[i]][idxs].to_value("kpc") 417 # Fix photon coordinates for regions crossing a periodic boundary 418 if ds.periodicity[i]:

File ~/.local/lib/python3.8/site-packages/unyt/array.py:1722, in unyt_array.getitem(self, item) 1721 def getitem(self, item): -> 1722 ret = super(unyt_array, self).getitem(item) 1723 if getattr(ret, "shape", None) == (): 1724 ret = unyt_quantity(ret, bypass_validation=True, name=self.name)

IndexError: index 1189003 is out of bounds for axis 0 with size 1048576

/home/rongjun/.local/lib/python3.8/site-packages/unyt/array.py(1722)getitem() 1720 1721 def getitem(self, item): -> 1722 ret = super(unyt_array, self).getitem(item) 1723 if getattr(ret, "shape", None) == (): 1724 ret = unyt_quantity(ret, bypass_validation=True, name=self.name)

---

---

In pdb, I navigate to "--> 416 pos = chunk[p_fields[i]][idxs].to_value("kpc")" I can see that chunk.shape' is (16777216,). That is good because 128*128*1024=16777216. Butchunk[p_fields[i]]' is actually only (1048576,) And `p_fields[i]' is ('index', 'x') as i=0 here So I don't know why the size of ('index', 'x') field reduces from 16777216 to 1048576 in make_photon function.

I also see that before using make_photon, sp_plt1555000[('index', 'x')].shape is still (16777217,), but after make_photon, it reduces to (1048576,)

[Rongjun-ANU]

I also found that, idxs will change once I rerun the make_photon function. So every time I use make_photon, it generates a different list of indexes and there is always some index greater than 1048576, and this returns indexerror.

[jzuhone]

Hk @Rongjun-ANU , can you send a full traceback and script? That will help.

[Rongjun-ANU]

Hi @jzuhone , thank you. I think that is the full traceback:

pyxsim : [INFO     ] 2023-10-12 15:24:41,381 Cosmology: h = 0.71, omega_matter = 0.27, omega_lambda = 0.73
pyxsim : [INFO     ] 2023-10-12 15:24:41,382 Using emission measure field '('gas', 'emission_measure')'.
pyxsim : [INFO     ] 2023-10-12 15:24:41,382 Using temperature field '('gas', 'temperature')'.

Preparing spectrum table : 100%
138/138 [00:22<00:00, 88.50it/s]
Processing cells/particles : 6%
16777216/268435456 [02:35<43:13, 97050.28it/s]

An error occurred: index 1134054 is out of bounds for axis 0 with size 1048576

Traceback (most recent call last):
  File "/tmp/ipykernel_929193/3620299636.py", line 15, in <module>
    n_photons, n_cells = pyxsim.make_photons("plt1555000_photon_list",
  File "/home/rongjun/.local/lib/python3.8/site-packages/pyxsim/photon_list.py", line 416, in make_photons
    pos = chunk[p_fields[i]][idxs].to_value("kpc")
  File "/home/rongjun/.local/lib/python3.8/site-packages/unyt/array.py", line 1722, in __getitem__
    ret = super(unyt_array, self).__getitem__(item)
IndexError: index 1134054 is out of bounds for axis 0 with size 1048576

Here is the link to my script: https://github.com/Rongjun-ANU/Indexerror/blob/main/Dash-Code.ipynb