cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes
https://ctapipe.readthedocs.org
BSD 3-Clause "New" or "Revised" License
63 stars 267 forks source link

Add option single_ob to HDF5Merger for merging chunks of the same ob #2436

Open maxnoe opened 10 months ago

maxnoe commented 10 months ago

Needed for merging of observations that are stored in multiple chunks into one final file for the OB.

TODO:

kosack commented 10 months ago

Is an option really needed for this? I would think the default behavior should just be updated to not duplicate an OB if it already exists.

maxnoe commented 10 months ago

Is an option really needed for this? I would think the default behavior should just be updated to not duplicate an OB if it already exists.

That could be done, yes. Depends a bit on the use case, e.g. we might want to check for uniqueness of obs_id and error out if not single_ob to prevent people from merging the same file twice or merging two simulation files that are different but have the same run number

codecov[bot] commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4a45dcd) 92.46% compared to head (4c334bf) 92.46%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2436 +/- ## ======================================= Coverage 92.46% 92.46% ======================================= Files 234 234 Lines 19868 19872 +4 ======================================= + Hits 18370 18374 +4 Misses 1498 1498 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.