cta-observatory / nectarchain

NectarCAM high level analysis tools
https://nectarchain.readthedocs.io
BSD 3-Clause "New" or "Revised" License
7 stars 18 forks source link

DQM output format and parsing script #147

Closed jlenain closed 2 months ago

jlenain commented 3 months ago

This PR introduces a new format for the DQM output in FITS files, as well as a user script which, once a DQM analysis has been run on DIRAC, grabs the results from DIRAC, extracts the archive and ingests the results in the ZODB data base.

This is a fundamental brick towards an automatic pipeline which would launch the DQM systematically on DIRAC on data transfer from CEA to DIRAC, and results ingestion in ZODB.

This was tested to run fine locally (see http://lpnhe-cta2.in2p3.fr/nectarcam-dqm), but we need to merge this PR, to trigger the building of an Apptainer image in CI, which would then be used to launch a couple of DQM jobs on DIRAC, to be able to further test the user script from start to finish.

In case of issues along the way, another PR will be opened.

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 14.54545% with 47 lines in your changes missing coverage. Please review.

Project coverage is 48.94%. Comparing base (b306f30) to head (119b3dd). Report is 23 commits behind head on main.

Files Patch % Lines
src/nectarchain/dqm/bokeh_app/main.py 0.00% 23 Missing :warning:
src/nectarchain/dqm/dqm_summary_processor.py 16.66% 20 Missing :warning:
src/nectarchain/dqm/start_dqm.py 0.00% 2 Missing :warning:
src/nectarchain/dqm/trigger_statistics.py 0.00% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #147 +/- ## ========================================== + Coverage 40.76% 48.94% +8.18% ========================================== Files 65 64 -1 Lines 4452 4566 +114 ========================================== + Hits 1815 2235 +420 + Misses 2637 2331 -306 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.