HERA-Team / hera_qm

HERA Data Quality Metrics
MIT License
2 stars 2 forks source link

Create `even_odd_zeros_checker()` to find packet loss and x-engine failures #424

Closed jsdillon closed 2 years ago

jsdillon commented 2 years ago

This PR introduces a new checker that looks for zeros in even or odd visibility data, given sum and diff data. This classifies antennas based on the maximum number of zeros in a spectrum attributable to that antenna (attribution always goes to the antenna in the baseline that participates in more visibility zeros).

The motivation is as follows (copy-pasted from #correlator in Slack):

Looking at a day that @elerath was trying to calibrate (2459820, August 28), I’m finding lots of zeros in the data that I didn’t expect when I look at odds and evens (i.e. sum - diff or sum + diff). A handful are just antennas registering zeros across the band. A bunch more have zeros in the even/odd autos (these appear to all be in Node 19) image But there’s at least one that’s showing zeros in its cross-correlations and not in its autos (102) image This is for a single file /lustre/aoc/projects/hera/jsdillon/2459820/zen.2459820.60032.sum.uvh5

codecov[bot] commented 2 years ago

Codecov Report

Merging #424 (49f86a5) into main (7d4aa18) will increase coverage by 0.01%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #424      +/-   ##
==========================================
+ Coverage   96.98%   96.99%   +0.01%     
==========================================
  Files          10       10              
  Lines        3444     3465      +21     
==========================================
+ Hits         3340     3361      +21     
  Misses        104      104              
Impacted Files Coverage Δ
hera_qm/ant_class.py 97.23% <100.00%> (+0.36%) :arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.