Error while processing the job

[edesfandiari]

Version of compliance checker running:

Describe the checker this affects: The ACDD, IOOS, and CF checkers are currently considered in scope for the purposes of the issue tracking on this repository. Issues with other plugins should be placed into their respective repositories.

Attach a minimal CDL or NetCDF file which is able to reproduce the issue

To Reproduce: List the steps or command line to run to reproduce the issue

Describe the issue below: Any additional detail or information relevant to the issue

[edesfandiari]

Recently, my attempts to check few short files using CF 1.6, 1.7, or 1.8 keep failing. The .nc files upload successfully but they all fail with "Error while processing the job. Job Timed Out" messages! My attempts even fails for same files that were successfully checked back in February. I'm using Linux RedHat and have the same issue using both Chrome and FireFox.

Also, the 'feedback' does not function either. Sending a note results in => compliance.ioos.us says An Error has Occurred

[edesfandiari]

I am using using https://compliance.ioos.us/index.html (using CF 1.8). Attached is a sample file .nc4 file (renamed by addig .csv to get it to upload) which was successful before. Also attached are screen shots of processing and timed out error (takes a long time after a quick and successful upload).

GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4.csv

[ocefpaf]

Looks like a problem with https://github.com/ioos/compliance-checker-web/, can you open an issue there? If you can make that file available I can try to local version for you and/or help you install it there.

[edesfandiari]

Note - The uploaded file GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4.csv is actually the same duplicate copy of GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4.nc.csv

[benjwadams]

Well, for one thing, Compliance Checker doesn't work on CSV currently to the best of my knowledge, even with formatting that IOOS recommends.

[ocefpaf]

I assume that was a copy and paste issue b/c the image above shows a different file. If you can share that file we can try to help you debug it.

[edesfandiari]

The uploaded file was a netcdf file. I used the "Paste, drop, or click to add files" at the bottom of the page to do my uploads. But, the .nc[4] is not among the allowed options. Therefore, I added a dummy .csv extension to my file to be able to upload it. However, I just noticed the attach option on top of the page and was not able to upload my .nc4 file. As mentioned before, the CF checker times out for all netcdf and hdf files. I also asked a colleague to try it and he had the same issue.

[edesfandiari]

I'm sorry but I can't attach any of my .nc, .nc4, .hdf. or .cdl files. I keep getting "We don't support that file type" as seen in this upload_options.png screenshot.

[edesfandiari]

As suggested, I also opened this issue, yesterday: https://github.com/ioos/compliance-checker-web/issues/121

[ocefpaf]

@edesfandiari can you zip the file? That way GH will allow you to upload it here.

[edesfandiari]

sure. Here it is. GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4.zip

[ocefpaf]

I was able to run it locally without a problem. The problem maybe be in the online version of compliance-checker.

compliance-checker --test=cf:1.8 GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4

Running Compliance Checker on the datasets from: ['GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4']

--------------------------------------------------------------------------------
                         IOOS Compliance Checker Report                         
                                 Version 5.1.0                                  
                     Report generated 2024-05-29T16:57:47Z                      
                                     cf:1.8                                     
http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html
--------------------------------------------------------------------------------
                               Corrective Actions                               
GiOCEAN_e1.aci_tavg_1dy_glo_L360x181_p27.19980201_1200z.nc4 has 7 potential issues

                                     Errors                                     
--------------------------------------------------------------------------------
§2.1 Filename
* Dataset path <bound method Dataset.filepath of <class 'compliance_checker.MemoizedDataset'>
root group (NETCDF4 data model, file format HDF5):
    History: File written by MAPL_CFIO
    Source: Icarus-3_3_p2
    Title: GiOCEAN_e1
    Contact: http://gmao.gsfc.nasa.gov
    Conventions: COARDS
    Institution: NASA Global Modeling and Assimilation Office
    References: http://gmao.gsfc.nasa.gov
    Comment: GiOCEAN_e1
    dimensions(sizes): lon(360), lat(181), lev(27), time(0)
    variables(dimensions): float64 lon(lon), float64 lat(lat), float64 lev(lev), int32 time(time), float32 CCN04(time, lev, lat, lon), float32 CDNC_NUC(time, lev, lat, lon), float32 CNVMFD(time, lev, lat, lon), float32 CNV_FICE(time, lev, lat, lon), float32 INC_NUC(time, lev, lat, lon), float32 NCPI_VOL(time, lev, lat, lon), float32 NCPL_VOL(time, lev, lat, lon), float32 NHET_IMM(time, lev, lat, lon), float32 NHET_NUC(time, lev, lat, lon), float32 PHIS(time, lat, lon), float32 SCF_ALL(time, lev, lat, lon), float32 WSUB(time, lev, lat, lon)
    groups: > must end with ".nc"

§3.3 Standard Name
* Standard name modifier "geopotential height" for variable PHIS is not a valid modifier according to CF Appendix C
* Standard name modifier "cloud droplet concentration (grid_avg)" for variable CDNC_NUC is not a valid modifier according to CF Appendix C
* Standard name modifier "Scale in-cloud vertical velocity" for variable WSUB is not a valid modifier according to CF Appendix C
* Standard name modifier "cloud fraction including snow and rain" for variable SCF_ALL is not a valid modifier according to CF Appendix C
* Standard name modifier "ice crystal concentration (grid_avg)" for variable INC_NUC is not a valid modifier according to CF Appendix C
* Standard name modifier "conc at 0.4 % supersaturation (grid_avg)" for variable CCN04 is not a valid modifier according to CF Appendix C
* Standard name modifier "ice crystal concentration by het freezing (grid_avg)" for variable NHET_NUC is not a valid modifier according to CF Appendix C
* Standard name modifier "fraction in convective tower" for variable CNV_FICE is not a valid modifier according to CF Appendix C
* standard_name particle_number_for_liquid_cloud is not defined in Standard Name Table v72. Possible close match(es): ['thickness_of_liquid_water_cloud']
* standard_name surface is not defined in Standard Name Table v72. Possible close match(es): ['source', 'surface_cover', 'surface_albedo']
* standard_name Nucleated is not defined in Standard Name Table v72.
* standard_name Subgrid is not defined in Standard Name Table v72.
* standard_name PLE_level is not defined in Standard Name Table v72.
* standard_name Supercooled is not defined in Standard Name Table v72.
* standard_name Nucleated is not defined in Standard Name Table v72.
* standard_name CCN is not defined in Standard Name Table v72.
* standard_name particle_number_for_ice_cloud is not defined in Standard Name Table v72.
* standard_name detraining_mass_flux is not defined in Standard Name Table v72. Possible close match(es): ['rainfall_flux']
* standard_name Nucleated is not defined in Standard Name Table v72.
* standard_name Ice is not defined in Standard Name Table v72.
* standard_name Immersion_IN is not defined in Standard Name Table v72.

§5 Coordinate Systems
* Coordinate variable "lon" must be strictly monotonic
* Coordinate variable "lat" must be strictly monotonic
* Coordinate variable "lev" must be strictly monotonic

                                    Warnings                                    
--------------------------------------------------------------------------------
§2.5 Variables
* CCN04:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* CDNC_NUC:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* CNVMFD:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* CNV_FICE:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* INC_NUC:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* NCPI_VOL:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* NCPL_VOL:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* NHET_IMM:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* NHET_NUC:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* PHIS:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* SCF_ALL:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)
* WSUB:_FillValue (999999986991104.0) should be outside the range specified by valid_range (-999999986991104.0, 999999986991104.0)

§2.6 Attributes
* §2.6.2 global attribute title should exist and be a non-empty string
* §2.6.2 global attribute history should exist and be a non-empty string
* §2.6.1 Conventions global attribute does not contain "CF-1.8"

§4.1 Latitude Coordinate
* latitude variable 'lat' should define standard_name='latitude' or axis='Y'

§4.2 Longitude Coordinate
* longitude variable 'lon' should define standard_name='longitude' or axis='X'

[edesfandiari]

Yes, that is correct. As previously stated, I've been using the online https://compliance.ioos.us/index.html

[ocefpaf]

Yes, that is correct. As previously stated, I've been using the online https://compliance.ioos.us/index.html

@edesfandiari we needed to be sure that it was not the file first and it seems you did not try the CLI, right?

Let's move this discussion to https://github.com/ioos/compliance-checker-web/issues/121 b/c here is nothing wrong with the code here.

[edesfandiari]

Correct, I have always used the online version and never the CLI.