Support standard intake online catalogs & PAVICS catalogs

Ouranosinc / xscen

A climate change scenario-building analysis framework.

https://xscen.readthedocs.io/

Apache License 2.0

13 stars 2 forks source link

Support standard intake online catalogs & PAVICS catalogs #15

Open RondeauG opened 2 years ago

RondeauG commented 2 years ago

Multiple intake-compatible catalogs already exist online, but can't be used yet in xscen (both the catalog and some of the functions that rely on specific column names.

The main issue for Data/ProjectCatalog is that most of them use a zstore column in lieu of path.
There would also be necessary changes elsewhere, probably, as some function require specific column names.

List of online catalogs : https://intake-esm.readthedocs.io/en/stable/supplemental-guide/faq.html#is-there-a-list-of-existing-catalogs

RondeauG commented 1 year ago

intake-esm appears to crash when attemping to open datasets defined in NcML files, such as is the case in PAVICS.

Note: pcat.to_dataset_dict() fails, but xr.open_dataset(pcat.df.iloc[0]["path"]) works, so it is on intake-esm's side rather than xarray. The error indicates that fsspec might also be to blame. We'll have to investigate.