Closed steven-murray closed 1 year ago
This looks quite reasonable (though I'm surprised by the level of speed-up you're seeing), but it's failing tests...
Base: 97.13% // Head: 97.18% // Increases project coverage by +0.04%
:tada:
Coverage data is based on head (
75cc6bf
) compared to base (9790053
). Patch coverage: 100.00% of modified lines in pull request are covered.
:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
This caches the result of
polstr2num
for a particular HERAData object because I was finding that when reading a single file from H6C, when getting the slices for each baseline, it took about 12 seconds, or ~20% of the total read time. When LST binning, this can be an even higher percentage because you might only read one out of the two times -- decreasing actual read time but not decreasing the number of baselines that need to be indexed.This PR makes the indexing time essentially negligible.
FWIW, on a single-shot read (the above was done reading 350 baselines at a time, similar to what would be done in a real pipeline), about 1/3 of the read time is taken in doing copies of the data into the datacontainers. I wonder if it would be useful to have a non-copying option.