HERA-Team / hera_cal

Library for HERA data reduction, including redundant calibration, absolute calibration, and LST-binning.
MIT License
13 stars 8 forks source link

Improve the speed of `smoothcal.solve_2d_DPSS` by adding additional filtering options in `nucal._linear_fit` #888

Closed tyler-a-cox closed 1 year ago

tyler-a-cox commented 1 year ago

This PR improves the speed of smoothcal.solve_2D_DPSS by using nucal._linear_fit to allow for additional filtering options. For the nominal inputs of smooth_cal.CalibrationSmoother.time_freq_2D_filter, the new lu_solve method is roughly 12x faster than the previously used pinv method.

For other uses of this function such as full_day_rfi.ipynb and full_data_auto_checker.ipynb where filtering scale is finer than the typical smooth_cal run, the new lu_solve method is significantly faster with comparable results.

Screenshot 2023-04-08 at 3 32 35 PM

This PR also deprecates the XTXinv keyword argument in favor of a more general cached_input, which also stores intermediate data products computed by scipy.linalg.lu_factor.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (ec4fcf6) 97.15% compared to head (a63ee4a) 97.15%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #888 +/- ## ======================================= Coverage 97.15% 97.15% ======================================= Files 20 20 Lines 9203 9214 +11 ======================================= + Hits 8941 8952 +11 Misses 262 262 ``` | [Impacted Files](https://codecov.io/gh/HERA-Team/hera_cal/pull/888?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team) | Coverage Δ | | |---|---|---| | [hera\_cal/nucal.py](https://codecov.io/gh/HERA-Team/hera_cal/pull/888?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team#diff-aGVyYV9jYWwvbnVjYWwucHk=) | `100.00% <100.00%> (ø)` | | | [hera\_cal/smooth\_cal.py](https://codecov.io/gh/HERA-Team/hera_cal/pull/888?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team#diff-aGVyYV9jYWwvc21vb3RoX2NhbC5weQ==) | `96.74% <100.00%> (+0.01%)` | :arrow_up: | ... and [2 files with indirect coverage changes](https://codecov.io/gh/HERA-Team/hera_cal/pull/888/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team) Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HERA-Team)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.