cedadev / cf-checker

Other
73 stars 31 forks source link

cf-checker crashes #89

Open larsbarring opened 3 years ago

larsbarring commented 3 years ago

The cf-checker crashes when checking what looks like a CF-compliant file (maybe with some quirks). Even if there is an obvious error that nevertheless escapes us, the checker should not crash. Below follows 1) output from ncdump -h 2) error output when using the Reading online tool 3) error output from running cf-checker locally installed.

ncdump -h

dimensions:
    time = UNLIMITED ; // (31 currently)
    y = 823 ;
    x = 567 ;
    nv = 2 ;
variables:
    float pr(time, y, x) ;
        pr:grid_mapping = "Lambert_Conformal" ;
        pr:long_name = "Total Precipitation" ;
        pr:param = "52.1.0" ;
        pr:units = "kg m**-2" ;
    double time(time) ;
        time:axis = "T" ;
        time:bounds = "time_bounds" ;
        time:calendar = "proleptic_gregorian" ;
        time:standard_name = "time" ;
        time:units = "hours since 1960-12-31 06:00:00" ;
    double x(x) ;
        x:axis = "X" ;
        x:standard_name = "projection_x_coordinate" ;
        x:units = "m" ;
    double y(y) ;
        y:axis = "Y" ;
        y:standard_name = "projection_y_coordinate" ;
        y:units = "m" ;
    double time_bounds(time, nv) ;
        time_bounds:axis = "T" ;
        time_bounds:bounds = "time_bounds" ;
        time_bounds:calendar = "proleptic_gregorian" ;
        time_bounds:standard_name = "time" ;
        time_bounds:units = "hours since 1960-12-31 06:00:00" ;
    int Lambert_Conformal ;
        Lambert_Conformal:grid_mapping_name = "lambert_conformal_conic" ;
        Lambert_Conformal:standard_parallel = 63. ;
        Lambert_Conformal:longitude_of_central_meridian = 15. ;
        Lambert_Conformal:latitude_of_projection_origin = 63. ;
        Lambert_Conformal:earth_radius = 6371229. ;
        Lambert_Conformal:longitudeOfFirstGridPointInDegrees = 6.586059 ;
        Lambert_Conformal:latitudeOfFirstGridPointInDegrees = 53.139963 ;
// global attributes:
        :_NCProperties = "version=1|netcdflibversion=4.4.1.1|hdf5libversion=1.10.1" ;
        :CDI = "Climate Data Interface version 1.9.8 (https://mpimet.mpg.de/cdi)" ;
        :Conventions = "CF-1.6" ;
        :CDO = "Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)" ;

Reading online tool

Output of CF-Checker follows...

CHECKING NetCDF FILE: /tmp/24345.nc
=====================
Using CF Checker Version 3.1.1
Checking against CF Version CF-1.6
Using Standard Name Table Version 77 (2021-01-19T13:38:50Z)
Using Area Type Table Version 10 (23 June 2020)
Using Standardized Region Name Table Version 4 (18 December 2018)

ERROR: (7.1): Incorrect number of dimensions for boundary variable: time_bounds
Traceback (most recent call last):
  File "/home/ros/anaconda2/bin/cfchecks", line 10, in 
    sys.exit(main())
  File "/home/ros/anaconda2/lib/python2.7/site-packages/cfchecker/cfchecks.py", line 3021, in main
    inst.checker(file)
  File "/home/ros/anaconda2/lib/python2.7/site-packages/cfchecker/cfchecks.py", line 488, in checker
    return self._checker()
  File "/home/ros/anaconda2/lib/python2.7/site-packages/cfchecker/cfchecks.py", line 659, in _checker
    (coordVars,auxCoordVars,boundsVars,climatologyVars,gridMappingVars)=self.getCoordinateDataVars()
  File "/home/ros/anaconda2/lib/python2.7/site-packages/cfchecker/cfchecks.py", line 1135, in getCoordinateDataVars
    if not (boundsData[i][0] <= value <= boundsData[i][1]) and not (boundsData[i][0] >= value >= boundsData[i][1]):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Locally installed cf-checker

Using CF Checker Version 4.0.0
Checking against CF Version CF-1.7
Using Standard Name Table Version 77 (2021-01-19T13:38:50Z)
Using Area Type Table Version 10 (23 June 2020)
Using Standardized Region Name Table Version 4 (18 December 2018)

WARN: (2.6.1): Inconsistency - This netCDF file appears to contain CF-1.6 data, but you've requested a validity check against CF-1.7
WARN: (7.1): Boundary var time_bounds should not have attribute units
WARN: (7.1): Boundary var time_bounds should not have attribute standard_name
WARN: (7.1): Boundary var time_bounds should not have attribute axis
WARN: (7.1): Boundary var time_bounds should not have attribute calendar
ERROR: (7.1): Incorrect number of dimensions for boundary variable: time_bounds
WARN: (7.1): Boundary var time_bounds should not have attribute units
WARN: (7.1): Boundary var time_bounds should not have attribute standard_name
WARN: (7.1): Boundary var time_bounds should not have attribute axis
WARN: (7.1): Boundary var time_bounds should not have attribute calendar
Traceback (most recent call last):
  File "/home/sm_lbarr/.conda/envs/cf-checker/bin/cfchecks", line 8, in <module>
    sys.exit(main())
  File "/home/sm_lbarr/.conda/envs/cf-checker/lib/python3.8/site-packages/cfchecker/cfchecks.py", line 3097, in main
    inst.checker(file)
  File "/home/sm_lbarr/.conda/envs/cf-checker/lib/python3.8/site-packages/cfchecker/cfchecks.py", line 532, in checker
    return self._checker()
  File "/home/sm_lbarr/.conda/envs/cf-checker/lib/python3.8/site-packages/cfchecker/cfchecks.py", line 705, in _checker
    (coordVars,auxCoordVars,boundsVars,climatologyVars,gridMappingVars)=self.getCoordinateDataVars()
  File "/home/sm_lbarr/.conda/envs/cf-checker/lib/python3.8/site-packages/cfchecker/cfchecks.py", line 1186, in getCoordinateDataVars
    if not (boundsData[i][0] <= value <= boundsData[i][1]) and not (boundsData[i][0] >= value >= boundsData[i][1]):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
larsbarring commented 3 years ago

And then the offending errors revealed the themselves: Of course, variable time_bounds must not have attributes axis = "T" and bounds = "time_bounds". Obvious once one sees them, but before that happens it would be helpful if the cf-checker would not crash and instead give a hint towards the problem.

RosalynHatcher commented 3 years ago

Thanks for reporting this @larsbarring. I'll put a fix in so it doesn't crash and add some indication as to the problem.

RosalynHatcher commented 3 years ago

Fixed so CF Checker no longer crashes in PR #91

Further work required to add better indication of the problem.