Unique errors for CF

[RuthPetrie]

['(2.3): Invalid variable name']
['(2.3): Invalid variable name', '(3.1): Invalid units: -']
['(2.6.3): Variable areacella named as an external variable must not be present in this file']
['(2.6.3): Variable volcello named as an external variable must not be present in this file']
['(3.1): Invalid units: -']
['(3.1): Units are not consistent with those given in the standard_name table.']
['(3.3): Invalid region name: a', '(3.3): Invalid region name: i', '(3.3): Invalid region name: g']
['(3.3): Invalid region name: atlantic_arctic_extended_ocean']
['(3.3): Invalid region name: carribean_windward_passage', '(3.3): Invalid region name: taiwan_and_luzon_straits', '(3.3): Invalid region name: agulhas_section']
['(3.3): Invalid region name: g', '(3.3): Invalid region name: a']
['(3.3): Invalid standard_name: air_pressure_at_top_of_atmosphere']
['(3.3): Invalid standard_name: alevel']
['(3.3): Invalid standard_name: atmosphere_optical_thickness_due_to_sulfate_ambient_aerosol']
['(3.3): Invalid standard_name: bounds']
['(3.3): Invalid standard_name: conductive_heat_flux_at_sea_ice_surface']
['(3.3): Invalid standard_name: depth_of_soil_moisture_saturation']
['(3.3): Invalid standard_name: heat_content_of_ocean_layer']
['(3.3): Invalid standard_name: Latitude', '(3.3): Invalid standard_name modifier: axis']
['(3.3): Invalid standard_name: ocean_sigma_z', '(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): No formula defined for standard name: ocean_sigma_z']
['(3.3): Invalid standard_name: ocean_sigma_z', '(4.3.3): No formula defined for standard name: ocean_sigma_z']
['(3.3): Invalid standard_name: olevel']
['(3.3): Invalid standard_name: sea_ice_basal_net_downward_sensible_heat_flux']
['(3.3): Invalid standard_name: sea_ice_salt_content']
['(3.3): Invalid standard_name: sea_ice_time_fraction']
['(3.3): Invalid standard_name: seaice_transect']
['(3.3): Invalid standard_name: tendency_of_sea_ice_amount_due_to_dynamics']
['(3.3): Invalid standard_name: tendency_of_sea_ice_amount_due_to_thermodynamics']
['(3.3): Invalid standard_name: tendency_of_snow_mass_due_to_sea_ice_dynamics']
['(3.3): Invalid standard_name: tendency_of_surface_snow_and_ice_amount_due_to_sublimation']
['(3.3): Invalid standard_name: total_water_storage']
['(3.3): Invalid standard_name: transect']
['(3.3): Invalid standard_name: Vertical', '(3.3): Invalid standard_name modifier: levels']
['(3.3): Invalid standard_name: vertical_integral_eastward_wind_by_dry_static_energy']
['(3.3): Invalid standard_name: vertical_integral_eastward_wind_by_total_water']
['(3.3): Invalid standard_name: vertical_integral_northward_wind_by_dry_static_energy']
['(3.3): Invalid standard_name: vertical_integral_northward_wind_by_total_water']
['(3.3): Invalid standard_name: water_flux_into_sea_water_from_sea_ice']
['(3.3): Invalid standard_name: water_flux_to_downstream']
["(3.3): Invalid syntax for 'standard_name' attribute: 'number of layer interfaces'"]
["(3.3): Invalid syntax for 'standard_name' attribute: 'number of layers'"]
['(3.3): Variable basin of invalid type. Region variable should be of type char.']
['(4.3.3): Formula term nsigma not present in formula for ocean_sigma_z_coordinate']
['(4.3.3): formula_terms attribute only allowed on coordinate variables']
['(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): ap_bnds is not declared as a variable', '(4.3.3): b_bnds is not declared as a variable', '(4.3.3): ps is not declared as a variable']
['(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): Formula term nsigma not present in formula for ocean_sigma_z_coordinate']
['(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): ps is not declared as a variable']
['(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): Standard name of variable ptop inconsistent with that of lev_bnds', '(4.3.3): Standard name of variable ps inconsistent with that of lev_bnds']
['(4.3.3): ps is not declared as a variable']
['(4.3.3): Standard name of variable ptop inconsistent with that of lev', '(4.3.3): Standard name of variable ps inconsistent with that of lev']
['(4.4.1): Attribute calendar may only be attached to time coordinate variable', '(3.1): Invalid units: days since', '(4.4): Invalid units and/or reference time']
['(5): coordinates attribute referencing non-existent variable']
['(5): co-ordinate variable not monotonic']
['(5): Dimensions must be a subset of dimensions of siarean']
['(5): Dimensions must be a subset of dimensions of siareas']
['(5): Dimensions must be a subset of dimensions of sivoln']
['(5): Dimensions must be a subset of dimensions of sivols']
['(5): Dimensions must be a subset of dimensions of thetaoga']
['(5): Dimensions must be a subset of dimensions of volo']
['(7.1): Boundary var lev_bnds has inconsistent standard_name to lev', '(3.1): Units are not consistent with those given in the standard_name table.', '(4.3.3): formula_terms attribute only allowed on coordinate variables']
['(7.1): Boundary var lev_bnds has inconsistent standard_name to lev', '(3.1): Units are not consistent with those given in the standard_name table.', '(4.3.3): formula_terms attribute only allowed on coordinate variables', '(4.3.3): p0 is not declared as a variable', '(4.3.3): a_bnds is not declared as a variable', '(4.3.3): b_bnds is not declared as a variable', '(4.3.3): ps is not declared as a variable']
['(7.1): Boundary var lev_bnds has inconsistent units to lev']
['(7.1): Boundary var time_bnds has inconsistent calendar to time']
['(7.1): Boundary var time_bnds has inconsistent units to time']
['(7.1): bounds attribute referencing non-existent variable alevel_bnds']
['(7.1): bounds attribute referencing non-existent variable lat_bnds']
['(7.1): bounds attribute referencing non-existent variable lon_bnds']
['(7.1): bounds attribute referencing non-existent variable sdepth_bnds']
['(7.1): Incorrect number of dimensions for boundary variable: time_bounds']
['(7.2): cell_measures variable areacella must either exist in this netCDF file or be named by the external_variables attribute']
['(7.2): cell_measures variable areacellg must either exist in this netCDF file or be named by the external_variables attribute']
['(7.2): cell_measures variable areacello must either exist in this netCDF file or be named by the external_variables attribute']
['(7.2): cell_measures variable areacello must either exist in this netCDF file or be named by the external_variables attribute', '(7.2): cell_measures variable volcello must either exist in this netCDF file or be named by the external_variables attribute']
['(7.2): Invalid cell_measures syntax']
['(7.3): Invalid cell_method: mask', "(7.3): Invalid 'name' in cell_methods attribute: (comment"]
["(7.3): Invalid 'name' in cell_methods attribute: month", "(7.3): Invalid 'name' in cell_methods attribute: year"]
["(7.3): Invalid 'name' in cell_methods attribute: scalar_axis"]
["(7.3): Invalid 'name' in cell_methods attribute: y", "(7.3): Invalid 'name' in cell_methods attribute: x"]
['(7.3): Invalid syntax for cell_methods attribute']
['(7.3): Invalid type1: landuse - must be a variable name or valid area_type']
['(7.3): Invalid type1: sector - must be a variable name or valid area_type']
['(7.3): Invalid unit hours, in cell_methods comment']
["Attribute missing_value of incorrect type (expecting 'Data Variable' type, got 'Numeric' type)"]
["Attribute missing_value of incorrect type (expecting 'Data Variable' type, got 'Numeric' type)", '(3.1): Units are not consistent with those given in the standard_name table.']
["Attribute missing_value of incorrect type (expecting 'Data Variable' type, got 'Numeric' type)", '(7.2): Invalid cell_measures syntax']
["Attribute missing_value of incorrect type (expecting 'Data Variable' type, got 'Numeric' type)", '(7.3): Invalid type1: sector - must be a variable name or valid area_type']
['Invalid attribute name: _CoordinateAxisType']

This is the list of unique errors. I haven't got counts, but when I load up the raw files into python, I do some aggregations and some counts, this was just a quick grep but it gives an idea of what we have.

[martinjuckes]

Thanks .. that is quite a list. Makes me wonder why there aren't any errors found for section 6 of CF. To make progress it would be useful to have the count of files and also perhaps a list of variable and models for associated with each error (both truncated if length > 20) and a few sample files.

(7.2): cell_measures ...... is one that I think we can flag as "dont care": this is a new CF rule brought in for CMIP6 to tidy up some issues which troubles the purists .. it shouldn't break anyone's software.

[martinjuckes]

PS: probably best to provide the above as json or csv.

[martinjuckes]

• missing_value errors discussed here: #3
• external_variables: see #4

[RuthPetrie]

50 : (2.3): Invalid variable name
147 : (2.6.3): Variable areacella named as an external variable must not be present in this file
1203 : (2.6.3): Variable volcello named as an external variable must not be present in this file
27 : (3.1): Invalid units: -
11049 : (3.1): Units are not consistent with those given in the standard_name table.
573 : (3.3): Invalid region name: a
489 : (3.3): Invalid region name: atlantic_arctic_extended_ocean
14 : (3.3): Invalid region name: carribean_windward_passage
70 : (3.3): Invalid region name: g
172 : (3.3): Invalid standard_name: air_pressure_at_top_of_atmosphere
2739 : (3.3): Invalid standard_name: alevel
52 : (3.3): Invalid standard_name: atmosphere_optical_thickness_due_to_sulfate_ambient_aerosol
2065 : (3.3): Invalid standard_name: bounds
101 : (3.3): Invalid standard_name: conductive_heat_flux_at_sea_ice_surface
10 : (3.3): Invalid standard_name: depth_of_soil_moisture_saturation
10 : (3.3): Invalid standard_name: heat_content_of_ocean_layer
114 : (3.3): Invalid standard_name: Latitude
3462 : (3.3): Invalid standard_name: ocean_sigma_z
1590 : (3.3): Invalid standard_name: olevel
101 : (3.3): Invalid standard_name: sea_ice_basal_net_downward_sensible_heat_flux
101 : (3.3): Invalid standard_name: sea_ice_salt_content
137 : (3.3): Invalid standard_name: sea_ice_time_fraction
101 : (3.3): Invalid standard_name: seaice_transect
101 : (3.3): Invalid standard_name: tendency_of_sea_ice_amount_due_to_dynamics
101 : (3.3): Invalid standard_name: tendency_of_sea_ice_amount_due_to_thermodynamics
101 : (3.3): Invalid standard_name: tendency_of_snow_mass_due_to_sea_ice_dynamics
244 : (3.3): Invalid standard_name: tendency_of_surface_snow_and_ice_amount_due_to_sublimation
33 : (3.3): Invalid standard_name: total_water_storage
101 : (3.3): Invalid standard_name: transect
2065 : (3.3): Invalid standard_name: Vertical
10 : (3.3): Invalid standard_name: vertical_integral_eastward_wind_by_dry_static_energy
10 : (3.3): Invalid standard_name: vertical_integral_eastward_wind_by_total_water
10 : (3.3): Invalid standard_name: vertical_integral_northward_wind_by_dry_static_energy
10 : (3.3): Invalid standard_name: vertical_integral_northward_wind_by_total_water
101 : (3.3): Invalid standard_name: water_flux_into_sea_water_from_sea_ice
3 : (3.3): Invalid standard_name: water_flux_to_downstream
51 : (3.3): Variable basin of invalid type. Region variable should be of type char.
764 : (4.3.3): Formula term nsigma not present in formula for ocean_sigma_z_coordinate
73020 : (4.3.3): formula_terms attribute only allowed on coordinate variables
238 : (4.3.3): ps is not declared as a variable
344 : (4.3.3): Standard name of variable ptop inconsistent with that of lev
10 : (4.4.1): Attribute calendar may only be attached to time coordinate variable
76 : (5): coordinates attribute referencing non-existent variable
15 : (5): co-ordinate variable not monotonic
126 : (5): Dimensions must be a subset of dimensions of siarean
126 : (5): Dimensions must be a subset of dimensions of siareas
126 : (5): Dimensions must be a subset of dimensions of sivoln
126 : (5): Dimensions must be a subset of dimensions of sivols
506 : (5): Dimensions must be a subset of dimensions of thetaoga
204 : (5): Dimensions must be a subset of dimensions of volo
2739 : (7.1): Boundary var lev_bnds has inconsistent standard_name to lev
914 : (7.1): Boundary var lev_bnds has inconsistent units to lev
8267 : (7.1): Boundary var time_bnds has inconsistent calendar to time
172 : (7.1): Boundary var time_bnds has inconsistent units to time
18 : (7.1): bounds attribute referencing non-existent variable alevel_bnds
4 : (7.1): bounds attribute referencing non-existent variable lat_bnds
4 : (7.1): bounds attribute referencing non-existent variable lon_bnds
20 : (7.1): bounds attribute referencing non-existent variable sdepth_bnds
12 : (7.1): Incorrect number of dimensions for boundary variable: time_bounds
10368 : (7.2): cell_measures variable areacella must either exist in this netCDF file or be named by the external_variables attribute
22 : (7.2): cell_measures variable areacellg must either exist in this netCDF file or be named by the external_variables attribute
2182 : (7.2): cell_measures variable areacello must either exist in this netCDF file or be named by the external_variables attribute
2997 : (7.2): Invalid cell_measures syntax
134 : (7.3): Invalid cell_method: mask
566 : (7.3): Invalid syntax for cell_methods attribute
1 : (7.3): Invalid type1: landuse - must be a variable name or valid area_type
644 : (7.3): Invalid type1: sector - must be a variable name or valid area_type
219 : (7.3): Invalid unit hours, in cell_methods comment
51893 : Data Variable
13492 : Invalid attribute name: _CoordinateAxisType

Just some initial numbers I'll get more details when I do proper analysis of the results.