DASDAE / dascore

A python library for distributed fiber optic sensing
Other
72 stars 16 forks source link

scan optimization #334

Closed d-chambers closed 8 months ago

d-chambers commented 8 months ago

Description

This PR implements an optimization for scan. Since in large DAS file archives a single file type is usually used, rather than iterate through each compatible FiberIO to figure out the format, for each file, once the format for a single file is discovered it is most likely all other files will be the same format. This PR prioritizes the last discovered format, then falls back to the normal behavior.

In the added benchmark, I observe speed ups of up to 100% in scanning a directory of files.

Checklist

I have (if applicable):

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d291976) 99.51% compared to head (33e41f0) 99.52%. Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #334 +/- ## ========================================== + Coverage 99.51% 99.52% +0.01% ========================================== Files 92 91 -1 Lines 7170 7172 +2 ========================================== + Hits 7135 7138 +3 + Misses 35 34 -1 ``` | [Flag](https://app.codecov.io/gh/DASDAE/dascore/pull/334/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/DASDAE/dascore/pull/334/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE) | `99.52% <100.00%> (+0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=DASDAE#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.