search

NOAA-GFDL / MDTF-diagnostics

Analysis framework and collection of process-oriented diagnostics for weather and climate simulations
https://mdtf-diagnostics.readthedocs.io/en/main/
Other
64 stars 100 forks source link

Slice requested time out of larger dataset if possible #716

Open jtmims opened 6 days ago

jtmims commented 6 days ago

What problem will this feature solve? Currently, the MDTF will fail for a given date-range even if it exists in a file in the catalog. This is due to the fact that the framework has no capability to grab a certain slice of the larger dataset and pass that along.

Describe the solution you'd like Around the check_group_daterange function, the framework should check to see if one of the queried files has the data wanted inside (i.e a 20yr file), slice the data wanted, and pass it along to the rest of the framework.

Describe alternatives you've considered

Additional context This problem was noticed during #714 . It would make for a good quality of life addition!

wrongkindofdoctor commented 6 days ago

@jtmims It looks like we need to re-implement the CropDateRange procedure after the data have been read into xarray. It doesn't necessarily have to be separate function as it was in prior versions, but it may be best to have it operate on each variable separately before doing the xarray merge.