openclimatefix / ocf_datapipes

OCF's DataPipe based dataloader for training and inference
MIT License
13 stars 11 forks source link

Work with lists of satellite inputs #173

Closed jacobbieker closed 1 year ago

jacobbieker commented 1 year ago

Pull Request

Description

Adds support for lists of satellite inputs, instead of wildcard.

This can be helpful for example, fsspec can be very slow when using the wildcard from GCP, but passing in a list of open_mfdataset is very quick.

How Has This Been Tested?

Unit tests

Checklist:

codecov[bot] commented 1 year ago

Codecov Report

Merging #173 (0612ba4) into main (c0679fd) will not change coverage. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #173   +/-   ##
=======================================
  Coverage   78.86%   78.86%           
=======================================
  Files         122      122           
  Lines        4967     4967           
=======================================
  Hits         3917     3917           
  Misses       1050     1050           
Impacted Files Coverage Δ
ocf_datapipes/load/satellite.py 94.44% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more