NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
120 stars 19 forks source link

CFA file incorrectly created with size 0 netCDF unlimited dimension #689

Closed davidhassell closed 1 year ago

davidhassell commented 1 year ago

At version 3.15.2, a CFA file can be incorrectly created with size 0 netCDF unlimited dimension, which causes a failure when the fiel is read:

>>> import cf
>>> f = cf.example_field(0)
>>> d = f.domain_axis('X')
>>> d.nc_set_unlimited(True)
>>> f.del_construct('X')
<CF DimensionCoordinate: longitude(8) degrees_east>
>>> cf.write(f, 'test.nc')
>>> g = cf.read('test.nc')
>>> cf.write(g, 'test.nca', cfa=True)
$ ncdump -h test.nca
netcdf test {
dimensions:
    lat = 5 ;
    bounds2 = 2 ;
    lon = UNLIMITED ; // (0 currently)
    f_2_loc = 2 ;
    f_1_loc = 1 ;
    f_lat = 1 ;
    f_lon = 1 ;
variables:
    double lat_bnds(lat, bounds2) ;
    double lat(lat) ;
        lat:units = "degrees_north" ;
        lat:standard_name = "latitude" ;
        lat:bounds = "lat_bnds" ;
    double time ;
        time:units = "days since 2018-12-01" ;
        time:standard_name = "time" ;
    double q ;
        q:project = "research" ;
        q:standard_name = "specific_humidity" ;
        q:units = "1" ;
        q:coordinates = "time" ;
        q:cell_methods = "area: mean" ;
        q:aggregated_dimensions = "lat lon" ;
        q:aggregated_data = "address: cfa_address file: cfa_file format: cfa_format location: cfa_location" ;
    int cfa_location(f_2_loc, f_1_loc) ;
    string cfa_file(f_lat, f_lon) ;
    string cfa_address ;
    string cfa_format ;

// global attributes:
        :Conventions = "CF-1.10 CFA-0.6.2" ;
}
>>> h = cf.read('test.nca')
Traceback (most recent call last):
    ...
ValueError: Can't set data: Input data must span all axes that have size greater than 1, as well as optionally spanning any size 1 axes.
>>>

The problem is that there is no actual (non-CFA) array to force the netCDF-C library to grow the unlimited dimension after it is created, and we only know the shape of the aggregated data from the netCDF dimensions, since the CFA variable is scalar.

The solution is to not write unlimited dimensions to CFA files. This makes sense, as they are not really inteded for dynamical growth, anyway. PR to follow.