HERA-Team / hera_qm

HERA Data Quality Metrics
MIT License
2 stars 2 forks source link

Check diff power by X-engine to look for mis-assigned packets #442

Closed jsdillon closed 1 year ago

jsdillon commented 1 year ago

This PR builds a new antenna classification method that looks of anomalously high power in the diffs which is hypothesized to come from stale or mis-assigned packets data being written to either the even or odd visibilities, leading sky signal in the diffs... and sometimes even autos leaking into cross sums.

This classifier computes the power of the diffs, relative to noise, converts that to a z-score, and then averages the magnitude of that z-score over X-engines. It then assigns "badness" to the minimal set of antennas that can explain all the baselines detected with anomalous power in the diffs.

See the discussion in Slack: https://eoranalysis.slack.com/archives/C3ZPGMG3E/p1686617551061159

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.02 :tada:

Comparison is base (1568d3d) 97.06% compared to head (ce86300) 97.09%.

:exclamation: Current head ce86300 differs from pull request most recent head 5b712eb. Consider uploading reports for the commit 5b712eb to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #442 +/- ## ========================================== + Coverage 97.06% 97.09% +0.02% ========================================== Files 11 11 Lines 3506 3540 +34 ========================================== + Hits 3403 3437 +34 Misses 103 103 ``` | [Impacted Files](https://app.codecov.io/gh/HERA-Team/hera_qm/pull/442?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team) | Coverage Δ | | |---|---|---| | [hera\_qm/ant\_class.py](https://app.codecov.io/gh/HERA-Team/hera_qm/pull/442?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team#diff-aGVyYV9xbS9hbnRfY2xhc3MucHk=) | `97.93% <100.00%> (+0.33%)` | :arrow_up: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.