search

cta-observatory / lstosa

On-site data processing pipeline for the LST-1
https://lstosa.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

Include R0V generation in OSA chain #298

Open contrera opened 1 month ago

contrera commented 1 month ago

Right now (summer 2024), R0V creation is done manually by the people at UAH, demanding a large amount of human work and introducing a bottleneck in the processing. We should consider the possibility of automatizing it.

In addition, AFAIK, no test is done that results are reproduced after R0V creation. Some automatic test on a subsample of data or a few variables would help to detect errors in R0V generation.

morcuended commented 1 month ago

I'm not familiar with the latest script for performing DVR. Is the one in #272 still being used? I guess it should be similar to the one for running gain selection.

moralejo commented 1 month ago

As for checks, you are right @contrera, they were done to validate on specific runs to validate the software, but there is no regular check. Basically it would be enough to check that (1) interleaved events have all pixels and both gains (2) shower events have all pixels that survived cleaning (those can be retrieved from the subrun-wise DL1 files, which contain the pixel masks), and of course just 1 gain (not sure if such a check is performed on R0G already)

morcuended commented 1 month ago

Plus the checks that jobs have been finished successfully and the number of files is consistent

(1) interleaved events have all pixels and both gains (2) shower events have all pixels that survived cleaning (those can be retrieved from the subrun-wise DL1 files, which contain the pixel masks), and of course just 1 gain (not sure if such a check is performed on R0G already)

I think these checks should be implemented in lstchain and called here