openclimatefix / ocf_datapipes

OCF's DataPipe based dataloader for training and inference
MIT License
13 stars 11 forks source link

fix time filtering for NWP dropout #217

Closed dfulu closed 1 year ago

dfulu commented 1 year ago

Pull Request

Description

The time filtering used previously was not properly taking into account the NWP dropout and the fact that it might select a previous timestamp as if the most recent NWP was not available. This caused some edge cases where the pipeline would fail.

This fixes the issue

Checklist:

codecov[bot] commented 1 year ago

Codecov Report

Merging #217 (690c62f) into main (ba5b318) will increase coverage by 0.00%. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #217   +/-   ##
=======================================
  Coverage   81.44%   81.44%           
=======================================
  Files         128      128           
  Lines        5568     5563    -5     
=======================================
- Hits         4535     4531    -4     
+ Misses       1033     1032    -1     
Files Changed Coverage Δ
ocf_datapipes/training/pvnet.py 82.22% <ø> (ø)
ocf_datapipes/training/common.py 95.54% <100.00%> (+0.47%) :arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more