search

XENONnT / straxen

Streaming analysis for XENON
BSD 3-Clause "New" or "Revised" License
21 stars 32 forks source link

Online veto intervals fix #1331

Open mflierm opened 6 months ago

mflierm commented 6 months ago

What does the code in this PR do / what does it improve?

A dedicated veto_intervals plugin for the online monitor is proposed. This is because sometimes at high rate calibrations, all processing fails just because the veto intervals can not be inserted in the MongoDB (max size 16MB).

Can you briefly describe how it works?

If the original veto_intervals chunk exceeds a certain size, the data is randomly sampled. To keep the cumulative deadtime info, an extra dtype is added that takes the cumulative deadtime from the original veto_intervals, but only the random indices are kept (this gives the same total cumulative deadtime). To avoid the database to overload after all again, the same random sampling is done if the new data array would exceed 10MB.

p.s. commit https://github.com/XENONnT/straxen/commit/c5e89bbe633a9a09f7cdf729baa44564a692febe was made by me (@mflierm), not by Joran (historical reasons).

coveralls commented 6 months ago

Coverage Status

coverage: 91.313% (-0.003%) from 91.316% when pulling 7073632644200575cb390fbdbef4f9f73a99769c on online_veto_intervals_fix into b18e8f32fc0b3049bdffd24fa36ea78e12593ff5 on master.