search

SWIFTSIM / swiftsimio

Python library for reading SWIFT data. Uses unyt and h5py.
GNU Lesser General Public License v3.0
15 stars 12 forks source link

Support for FOF files #156

Open MatthieuSchaller opened 1 year ago

MatthieuSchaller commented 1 year ago

We could probably add support for the FOF files rather easily. They have the same format as the snapshots but don't have /PartType?. They instead have Groups. The arrays within there have the same unit meta-data and description as the particle-carried fields. Only other difference is that the arrays have size /Header/NumGroups_Total.

JBorrow commented 10 months ago

This is a good first issue if you have any new grad students joining this year :)

MatthieuSchaller commented 10 months ago

@robjmcgibbon might also be interested as a first side thing to understand the internal structure of swiftsimio.

JBorrow commented 10 months ago

Sounds good. If Rob is interested, I am happy to help.

robjmcgibbon commented 10 months ago

Yep I think this would be good to work on to gain some understanding of the repo. Do you think it's worth having a call about how to go about it or should I just get stuck in and look myself?

JBorrow commented 10 months ago

Excellent! Feel free to take a look yourself first, it should be somewhat well documented. If it's not, then that's something we should address!

Ideally, the load function would figure out if you are trying to load a snapshot or a FoF output automatically.

All the code you commit to the repo will be reviewed in a similar manner to SWIFT itself. Let me know if you have any issues, and we can schedule a call, but feel free to get stuck in!

JBorrow commented 9 months ago

@robjmcgibbon Have you had a chance to look at this? Let me know if you're stuck!

Given how big the volumes are getting, it would be nice to have this work with partial spatial loading too...

robjmcgibbon commented 8 months ago

Hi Josh, nope haven't looked yet, been working on SOAP stuff. Hopefully will get to it next week when cosma is busy.

MatthieuSchaller commented 8 months ago

Note that I am looking at adding a meta "snapshot" too for the FOF catalogs on the SWIFT side. Might help here too.

MatthieuSchaller commented 7 months ago

FYI, there is now a meta-snapshot also for the FOF output. So only need to read one file even when running over MPI.