cta-observatory / cta-lstchain

LST prototype testbench chain
https://cta-observatory.github.io/cta-lstchain/
BSD 3-Clause "New" or "Revised" License
26 stars 77 forks source link

AssertionError in data_quality.ipynb #1277

Closed katagirihide closed 4 months ago

katagirihide commented 4 months ago

When I tried to apply data_quality.ipynb I got from github today to the data in 2021-present as follows:

files = glob.glob("/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/DL1_datacheck_202[1-4]*.h5") # since 2021-

the assertion error occurred like this:

' # Check that now the runsummary entries match the runs in runlist (in the same order)'
assert(np.allclose(runlist, runsummary['runnumber']))

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[20], line 2
      1 # Check that now the runsummary entries match the runs in runlist (in the same order)
----> 2 assert(np.allclose(runlist, runsummary['runnumber']))

AssertionError: 

When I tried with

files = glob.glob("/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/DL1_datacheck_202[4]*.h5") 

or

files = glob.glob("/fefs/aswg/data/real/OSA/DL1DataCheck_LongTerm/night_wise/all/DL1_datacheck_202[1-3]*.h5") 

, they worked well.

At the present, there are two faulty runs like:

Removing run 8091 from runsummary table!
Removing run 17855 from runsummary table!,

But run 17855 does not seem to be removed from runsummary. Since the lengths of runlist and runsummary table are the same, the different run may be removed.

image

moralejo commented 4 months ago

Fixed in #1279