Some updates for new CDS beta backend API

xcube-dev / xcube-cds

An xcube plugin to generate data cubes from the Climate Data Store (CDS) API

MIT License

3 stars 1 forks source link

Some updates for new CDS beta backend API #85

Closed pont-us closed 2 months ago

pont-us commented 4 months ago

Closes #84 .

These changes don't yet provide full support for the new API version (in part because the API server is in beta and it's not yet clear whether some of its new behaviours are planned changes or temporary bugs), but with them applied all but one of the unit tests pass. test_open_data_null_variables_list fails because the API server returns a Zip rather than the requested NetCDF.

Require cdsapi >=0.7.0
Add an api_version parameter to ERA5DatasetHandler.init (defaulting to 1).
Support new data_format parameter (replaces old format) in ERA5DatasetHandler
In CDSDataOpener, support both old and new ERA5 time coordinate names (time and valid_time respectively).

konstntokas commented 2 months ago

The open parameter spatial_res is changed from a required to an optional parameter for ERA5. The default grid size is then returned. This allows to request larger datasets.

konstntokas commented 2 months ago

I tried to invoke an error, when the dimension do not fit. However, the examples which produced errors last week, work fine now. I even added an example for ERA5, where I used parameters from atmosphere and ocean. The grid resolution of the ocean parameters is coarser by a factor of two. xarray.open_mfdataset() can deal with this, since the coarser grid of the ocean parameter is a subset of the finer grid of the atmospheric parameter. The missing values are filled with nans.

So far, I could not find an example, which gives me an error. I remember that xarray.open_mfdataset() gave an understandable error, when the dimensions do not fit. I would say, it is enough to rely on this, or maybe add a better error message, when an actual example comes up. At the moment I have the feeling the CDS-beta is evolving quickly.

pont-us commented 2 months ago

@konstntokas I agree that it's enough if things are working with the current backend state and our current tests and use cases. We'll deal with any new errors or problems as and when they come up. Test suite and notebooks have now finally run to completion and code looks fine, so I'll approve and merge.