European-XFEL / karabo_data

Python tools to read and analyse data from European XFEL
https://karabo-data.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
13 stars 7 forks source link

Validate index dataset size #211

Closed takluyver closed 5 years ago

takluyver commented 5 years ago

@RobertRosca was having problems with a run directory - I looked into it and found that some of the indexes had fewer entries than the train IDs. As far as I know, this is not meant to happen, so this adds a check for that (along with a couple of other scenarios which we haven't seen, but which would definitely cause problems if they did).

$ karabo-data-validate /gpfs/exfel/exp/SPB/201802/p002157/proc/r0045
...
Index has wrong number of entries
  dataset: INDEX/SPB_DET_AGIPD1M-1/DET/11CH0:xtdf/image
  file: ./CORR-R0045-AGIPD11-S00000.h5
  index_shape: (119,)
  trainids_shape: (256,)
RobertRosca commented 5 years ago

LGTM 👍