SWIFTSIM / swiftsimio

Python library for reading SWIFT data. Uses unyt and h5py.
GNU Lesser General Public License v3.0
15 stars 12 forks source link

Writing subset of FLAMINGO snapshot fails. #189

Closed VictorForouhar closed 1 month ago

VictorForouhar commented 1 month ago

I am trying to write a subset of a FLAMINGO snapshot, in particular, the particles contained within a specified swift cell. However, when I execute subset_writer.write_subset, the following error appears:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-73-c96a5989aa59> in <module>
----> 1 sw.subset_writer.write_subset("/snap8/scratch/dp004/dc-foro1/hbt_runs_latest/debug_fof/fof_subset_testing.hdf5", mask)

~/env/database/lib/python3.6/site-packages/swiftsimio/subset_writer.py in write_subset(output_file, mask)
    324     list_of_links, list_of_link_paths = find_links(infile)
    325     write_metadata(infile, outfile, list_of_links, mask)
--> 326     write_datasubset(infile, outfile, mask, find_datasets(infile), list_of_links)
    327     connect_links(outfile, list_of_links, list_of_link_paths)
    328 

~/env/database/lib/python3.6/site-packages/swiftsimio/subset_writer.py in write_datasubset(infile, outfile, mask, dataset_names, links_list)
    266             output_type = first_value.dtype
    267             output_size = first_value.size
--> 268             mask_size = get_dataset_mask(mask, name, suffix="_size")
    269             if output_size != 1:
    270                 output_shape = (mask_size, output_size)

~/env/database/lib/python3.6/site-packages/swiftsimio/subset_writer.py in get_dataset_mask(mask, dataset_name, suffix)
     67 
     68     if "PartType" in dataset_name:
---> 69         part_type = [int(x) for x in filter(str.isdigit, dataset_name)][0]
     70         mask_name = metadata.particle_types.particle_name_underscores[part_type]
     71         return getattr(mask, f"{mask_name}{suffix}", None)

IndexError: list index out of range
VictorForouhar commented 1 month ago

Code to reproduce :

snapshot_path = '/cosma8/data/dp004/flamingo/Runs/L1000N3600/HYDRO_FIDUCIAL/snapshots/flamingo_0066/flamingo_0066.hdf5'
mask = sw.mask(snapshot_path)

cell_centre = np.array([17, 26, 29], dtype=int)
cell_edges = np.array([[x, x + 1] for x in cell_centre])
load_region = cell_edges * mask.cell_size[0]

mask.constrain_spatial(load_region)
sw.subset_writer.write_subset(f"{output_file_path}.hdf5", mask)
MatthieuSchaller commented 1 month ago

Maybe naive suspicion: the neutrinos are PartType6.

JBorrow commented 1 month ago

Mm seems like it's failing on line 69, indicating a problem with the extraction of the particle type number, not its association with a string.

JBorrow commented 1 month ago

Have you made sure that you are running the latest version of swiftsimio? I recall us having this problem before, maybe we fixed it?

JBorrow commented 1 month ago

Yeah; your code works for me. I think you must just be running an outdated version of swiftsimio.

VictorForouhar commented 1 month ago

Ugh, let me update swiftsimio and confirm.

VictorForouhar commented 1 month ago

Okay, updated to latest version from my previous one (4.9.0) and it works! Thank you :)