cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes
https://ctapipe.readthedocs.org
BSD 3-Clause "New" or "Revised" License
63 stars 267 forks source link

TableLoader does not check uniqueness pre-conditions #2426

Open maxnoe opened 10 months ago

maxnoe commented 10 months ago

Describe the bug

TableLoader has uniqueness pre-conditions when merging the different tables, mainly on sb_id in the scheduling blocks, obs_id in the observation_blocks and the event primary keys in the event tables.

We don't check if these pre-conditions are met and astropy table joining explodes the rows in case of non-unique rows.

To Reproduce

Merge the same file twice using ctapipe-merge, read it with table loader.

Expected behavior

Some kind of user-friendly error