RuntimeError: NetCDF: Not a valid data type or _FillValue type mismatch

[mhadizadeh91]

Hi dear, I've got the following error when I want to run the code in the below:

import icclim from icclim.util import callback my_indice_params = {'indice_name': 'my_indice', 'calc_operation': 'nb_events', 'logical_operation': 'lt', 'thresh': 0 + 273.15 } file_tas = "/home/2002-T2max.nc" out_f = "my_indice.nc" icclim.indice(user_indice=my_indice_params, in_files=file_tas, var_name='tas', slice_mode='year', out_unit='days', out_file=out_f, callback=callback.defaultCallback2)

2021-09-14 15:26:48,744 **** 2021-09-14 15:26:48,744 2021-09-14 15:26:48,745 icclim V4.2.19 2021-09-14 15:26:48,745 2021-09-14 15:26:48,745 2021-09-14 15:26:48,745 Tue Sep 14 15:26:48 2021 GMT 2021-09-14 15:26:48,745 2021-09-14 15:26:48,745 BEGIN EXECUTION 2021-09-14 15:26:48,745 2021-09-14 15:26:48,745 **** Traceback (most recent call last): File "", line 1, in File "/home/icclim/icclim/icclim.py", line 175, in indice indice_dim = util_nc.copy_var_dim(inc, onc, var_name[0], lev_dim_pos=lev_dim_pos) # tuple ('time', 'lat', 'lon')
File "/home/icclim/icclim/util/util_nc.py", line 233, in copy_var_dim onc_dim0 = onc.createVariable( str(time_var), inc_dim0.dtype, (str(time_var)) ) # time File "src/netCDF4/_netCDF4.pyx", line 2748, in netCDF4._netCDF4.Dataset.createVariable File "src/netCDF4/_netCDF4.pyx", line 3808, in netCDF4._netCDF4.Variable.init File "src/netCDF4/_netCDF4.pyx", line 1927, in netCDF4._netCDF4._ensure_nc_success RuntimeError: NetCDF: Not a valid data type or _FillValue type mismatch

Also ncdump result is as follow:

$ ncdump -h ../2002-T2max.nc

netcdf \2002-T2max { dimensions: time = 365 ; lat = 121 ; lon = 121 ; variables: int64 time(time) ; time:long_name = "time" ; time:standard_name = "time" ; time:axis = "T" ; time:stored_direction = "increasing" ; time:type = "double" ; time:units = "days since 2002-01-01 00:00:00" ; time:calendar = "proleptic_gregorian" ; int64 realization ; realization:long_name = "realization" ; realization:units = "1" ; realization:standard_name = "realization" ; realization:stored_direction = "increasing" ; realization:type = "integer" ; double lat(lat) ; lat:_FillValue = NaN ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; lat:long_name = "latitude" ; lat:stored_direction = "decreasing" ; lat:axis = "Y" ; lat:positive = "up" ; lat:type = "double" ; lat:valid_max = 90. ; lat:valid_min = -90. ; double lon(lon) ; lon:_FillValue = NaN ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; lon:long_name = "longitude" ; lon:axis = "X" ; lon:positive = "up" ; lon:type = "double" ; lon:valid_max = 360. ; lon:valid_min = -180. ; float tas(time, lat, lon) ; tas:_FillValue = NaNf ; tas:long_name = "Near-Surface Air Temperature" ; tas:units = "K" ; tas:standard_name = "air_temperature" ; tas:comment = "near-surface (usually, 2 meter) air temperature" ; tas:cds_magics_style_name = "near-surface-air-temperature" ; tas:type = "real" ; tas:coordinates = "realization" ;

// global attributes: :Conventions = "CF-1.7" ; :institution = "European Centre for Medium-Range Weather Forecasts" ; :history = "2021-09-13T10:14 GRIB to CDM+CF via cfgrib-0.9.9.0/ecCodes-2.16.0 with {\"source\": \"/cache/data0/e27cfa3c612c16349f2e2306e1fbc570.grib\", \"filter_by_keys\": {}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ; :source = "ECMWF" ; }

Also my netcdf file has been attached.

2002-T2max.zip

Cheers,

[mhadizadeh91]

I changed 'time' to double type and gave all variables corresponding '_FillValue' but ultimately I received new error:

2021-09-14 19:07:27,682 **** 2021-09-14 19:07:27,682 2021-09-14 19:07:27,683 icclim V4.2.19 2021-09-14 19:07:27,683 2021-09-14 19:07:27,683 2021-09-14 19:07:27,683 Tue Sep 14 19:07:27 2021 GMT 2021-09-14 19:07:27,683 2021-09-14 19:07:27,683 BEGIN EXECUTION 2021-09-14 19:07:27,684 2021-09-14 19:07:27,684 **** Traceback (most recent call last): File "test.py", line 6, in icclim.indice(user_indice=my_indice_params, in_files=file_tas, var_name='tas', slice_mode='year', out_unit='days', out_file=out_f, callback=callback.defaultCallback2) File "/home/mostafa/Bijandi_thesis/icclim/icclim/icclim.py", line 175, in indice indice_dim = util_nc.copy_var_dim(inc, onc, var_name[0], lev_dim_pos=lev_dim_pos) # tuple ('time', 'lat', 'lon')
File "/home/mostafa/Bijandi_thesis/icclim/icclim/util/util_nc.py", line 250, in copy_var_dim onc_dim0.setattr(attr[0], attr[1]) File "src/netCDF4/_netCDF4.pyx", line 4309, in netCDF4._netCDF4.Variable.setattr AttributeError: _FillValue attribute must be set when variable is created (using fill_value keyword to createVariable)

Also my ncdump is as follow:

$ ncdump -h 2002-T2max.nc

netcdf \2002-T2max { dimensions: time = 366 ; lat = 121 ; lon = 121 ; variables: double time(time) ; time:axis = "T" ; time:calendar = "proleptic_gregorian" ; time:long_name = "time" ; time:standard_name = "time" ; time:stored_direction = "increasing" ; time:type = "double" ; time:units = "days since 2000-01-01 00:00:00" ; time:_FillValue = -9.99999979021477e+33 ; int64 realization ; realization:long_name = "realization" ; realization:units = "1" ; realization:standard_name = "realization" ; realization:stored_direction = "increasing" ; realization:type = "integer" ; realization:_FillValue = -999 ; double lat(lat) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; lat:long_name = "latitude" ; lat:stored_direction = "decreasing" ; lat:axis = "Y" ; lat:positive = "up" ; lat:type = "double" ; lat:valid_max = 90. ; lat:valid_min = -90. ; lat:_FillValue = -9.99999979021477e+33 ; double lon(lon) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; lon:long_name = "longitude" ; lon:axis = "X" ; lon:positive = "up" ; lon:type = "double" ; lon:valid_max = 360. ; lon:valid_min = -180. ; lon:_FillValue = -9.99999979021477e+33 ; float tas(time, lat, lon) ; tas:long_name = "Near-Surface Air Temperature" ; tas:units = "K" ; tas:standard_name = "air_temperature" ; tas:comment = "near-surface (usually, 2 meter) air temperature" ; tas:cds_magics_style_name = "near-surface-air-temperature" ; tas:type = "real" ; tas:coordinates = "realization" ; tas:_FillValue = 1.e+36f ;

// global attributes: :Conventions = "CF-1.7" ; :institution = "European Centre for Medium-Range Weather Forecasts" ; :source = "ECMWF" ; :nco_openmp_thread_number = 1 ;

Edited file has been attached 2002-T2max.zip .

[pagecp]

I have check the file.

It seems that the netcdf-python does not support int64. Also _FillValue for a coordinate variable is not supported. So what I did to make it work was to:

• Remove _FillValue attributes from time lat lon
• Remove tas:coordinates attribute as realization is not a coordinate for tas
• Remove realization variable as it was not needed

[bzah]

With icclim 5.0.0 the copying of variable dimension is no longer done directly by icclim but instead we let xarray handle this kind of work. I'm closing this because if the issue still exist the error will be very different.