openclimatefix / ocf_datapipes

OCF's DataPipe based dataloader for training and inference
MIT License
13 stars 11 forks source link

Add ICON support #216

Closed jacobbieker closed 1 year ago

jacobbieker commented 1 year ago

Pull Request

Description

This PR focuses on decoupling datapipes so closely to UKV for NWP inputs, and adds support for ICON NWPs as well.

Fixes

How Has This Been Tested?

Unit tests

Checklist:

codecov[bot] commented 1 year ago

Codecov Report

Merging #216 (83bc69d) into main (ca78491) will decrease coverage by 0.12%. Report is 58 commits behind head on main. The diff coverage is 76.52%.

@@            Coverage Diff             @@
##             main     #216      +/-   ##
==========================================
- Coverage   81.22%   81.11%   -0.12%     
==========================================
  Files         128      131       +3     
  Lines        5508     5586      +78     
==========================================
+ Hits         4474     4531      +57     
- Misses       1034     1055      +21     
Files Changed Coverage Δ
ocf_datapipes/load/nwp/providers/gfs.py 35.13% <ø> (ø)
ocf_datapipes/select/select_spatial_slice.py 77.41% <58.69%> (-9.25%) :arrow_down:
ocf_datapipes/utils/geospatial.py 79.16% <66.66%> (-0.84%) :arrow_down:
ocf_datapipes/load/nwp/nwp.py 91.66% <75.00%> (-6.07%) :arrow_down:
ocf_datapipes/select/location_picker.py 95.00% <80.00%> (-2.15%) :arrow_down:
ocf_datapipes/load/nwp/providers/utils.py 83.33% <83.33%> (ø)
ocf_datapipes/load/nwp/providers/icon.py 84.61% <84.61%> (ø)
ocf_datapipes/load/__init__.py 81.25% <100.00%> (ø)
ocf_datapipes/load/nwp/providers/ukv.py 100.00% <100.00%> (ø)
ocf_datapipes/select/select_channels.py 100.00% <100.00%> (ø)
... and 1 more

... and 2 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more