search

jbusecke / pangeo-forge-esgf

Using queries to the ESGF API to generate urls and keyword arguments for receipe generation in pangeo-forge
Apache License 2.0
6 stars 4 forks source link

Determine time chunks based on calendar #11

Closed duncanwp closed 11 months ago

duncanwp commented 1 year ago

Current daily data processing (as introduced in https://github.com/jbusecke/pangeo-forge-esgf/pull/9) has to make some pretty rough assumptions about sensible time chunks.

The current chunking uses a mixture of 360-day and Julien years and can only assume 30 day months. It would be more user friendly to chunk into calendar months and years as per the underling calendar.

As far as I can tell the ESGF metadata doesn't include calendar information and this would have to be discerned from the time coordinate metadata in each file. I think it's safe to assume the calendar is the same for multiple files with the same ID, but will differ by model and may well differ between experiments and activities for each model.

jbusecke commented 11 months ago

Linking https://github.com/pangeo-forge/pangeo-forge-recipes/issues/572#issue-1862140876 here. Once that is implemented this could be a very useful rechunking 'plugin'!

jbusecke commented 11 months ago

Also closing this in favor of moving discussions over to PGF-recipes if that is ok.