search

aodn / python-aodntools

Repository for templates and code relating to generating standard NetCDF files for the Australia Ocean Data Network
GNU Lesser General Public License v3.0
10 stars 3 forks source link

float type 16 not recognised #46

Closed lbesnard closed 3 years ago

lbesnard commented 5 years ago

The template suggests that it is possible to use 'float16' https://github.com/aodn/aodn-netcdf-tools/blob/c62ba047f9aec92c340c71b444b9fe31250feb37/ncwriter/template_schema.json#L57

However, the netcdf-writer bugs for me for float16. (float, float32 and float64 work fine)

  File "/home/github_repo/chef/src/python-aodndata/python-aodndata-virtualenv/local/lib/python2.7/site-packages/ncwriter/template.py", line 305, in create_variables
    ncvar = self.ncobj.createVariable(varname, **var_c_opts)
  File "netCDF4/_netCDF4.pyx", line 2557, in netCDF4._netCDF4.Dataset.createVariable
  File "netCDF4/_netCDF4.pyx", line 3490, in netCDF4._netCDF4.Variable.__init__
TypeError: illegal primitive data type, must be one of ['i8', 'f4', 'f8', 'S1', 'i2', 'i4', 'u8', 'u4', 'u1', 'u2', 'i1'], got float16
ocehugo commented 5 years ago

AFAICR, there is no exact representation for float16 in C. The closest equivalent is short but a bit better than float16.

ggalibert commented 5 years ago

I'm not too fussed about supporting float 16, we barely ever use them. Float 32 though is more important.

mhidas commented 5 years ago

Ok, my bad. It turns out this is "false advertising" in the template schema. NetCDF actually doesn't support a 16-bit floating point type (see https://www.unidata.ucar.edu/software/netcdf/docs/data_type.html).

So the fix is to review all the values specified there, and remove any that are actually not consistent with netCDF.