openclimatefix / ocf_datapipes

OCF's DataPipe based dataloader for training and inference
MIT License
13 stars 11 forks source link

remove start-end datetime mixin #264

Closed dfulu closed 8 months ago

dfulu commented 8 months ago

Pull Request

Description

Remove the start/end time mixin as it is generally unused. The preferred way to limit the time range of samples is something like this

Note:

Initial discussions around this were that this might be less memory efficient, and that we'd be keeping unused data around by not filtering. However, based on this stack overflow question, I'm not sure we were even freeing up memory with the slicing we were using previously

Checklist:

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (287be67) 78.22% compared to head (9ccd8c9) 78.20%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #264 +/- ## ========================================== - Coverage 78.22% 78.20% -0.03% ========================================== Files 121 121 Lines 5571 5551 -20 ========================================== - Hits 4358 4341 -17 + Misses 1213 1210 -3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.