search

NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
119 stars 19 forks source link

CFAPyX Support - Complete groups in CFA #798

Open dwest77a opened 1 month ago

dwest77a commented 1 month ago

The Xarray-implemented CFA-netCDF reader has an issue with groups because Xarray does not support the typical group hierarchy of netCDF4. Xarray is capable of opening netCDF4 groups as separate Datasets but this requires the group to be 'complete' i.e self-describing in terms of dimension/coordinate variables. A set of groups which share a global dimension would therefore not be allowed with the current implementation.

Possible solutions to this issue are:

  1. Ensuring CFA-netCDF groups are complete i.e aggregation groups contain all the data and metadata required to serve a specific variable (time/lat/lon would all be part of the group)

  2. Adjusting the CFAPyX (Xarray-CFA implementation) package to expand groups out into their respective variables, then using either a variable attribute or other form to represent where the variable originated from in the source data.

dwest77a commented 1 month ago

I'll get to work testing option 2 to see if it is viable, but otherwise we will need to consider option 1, or accept that CFA-netCDF groups will not work with xarray.

davidhassell commented 1 month ago

Er, is this a cf-python issue :)