HERA-Team / hera_cal

Library for HERA data reduction, including redundant calibration, absolute calibration, and LST-binning.
MIT License
13 stars 8 forks source link

Fix inpaint nonfiring #921

Closed steven-murray closed 8 months ago

steven-murray commented 8 months ago

I think there were two things wrong here. So the way the code works is that it makes data and nsamples masked arrays. The masks are inpaint-aware flags (if inpaint mode is one), but the flags array is just the raw flags. After setting these masks, I was doing the flag thresholding, which set some new flags based on thresholds over the days for the old flags, but then I reset the masks to these new updated flags, which include all the raw flags. So I've fixed that.

Another thing that was happening is that the std was using the inpaint-aware flags, but it should be using the raw flags (so a completely flagged (but inpainted) integration should have a mean value, but std=inf.

I think the code already naturally handles the case where all the days are flagged (for a particular lst/freq/pol) in inpainted mode by giving it a non-nan average, so the discrepancy there was probably due to the other bugs.

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d89aaff) 97.17% compared to head (7de3c49) 97.17%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #921 +/- ## ======================================= Coverage 97.17% 97.17% ======================================= Files 23 23 Lines 10488 10490 +2 ======================================= + Hits 10192 10194 +2 Misses 296 296 ``` | [Flag](https://app.codecov.io/gh/HERA-Team/hera_cal/pull/921/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/HERA-Team/hera_cal/pull/921/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team) | `97.17% <100.00%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.