SUEPPhysics / SUEPCoffea_dask

SUEP analysis using coffea with fastjet. Uses Dask for batch submissions
3 stars 13 forks source link

Output size #267

Closed lucalavezzo closed 9 months ago

lucalavezzo commented 10 months ago

DO NOT MERGE THIS BEFORE MAKING GGF OFFLINE TAG -- SOME THINGS MAY NOT BE BACKWARDS COMPATIBLE

Metadata doesn't seem to be stable in pandas + hdf5. Instead store metadata as a separate dataframe in the same hdf5 file.

Addresses #265

Decided to stick with 'old way' to store metadata, at least for now, see the discussion in the issue. This PR has been modified to reduce a bit the size of the output via type casting.