inconsistent naming of pressure level variables not passing PrePARE

[oloapinivad]

I have a few variables cmorized with the Jon Seddon tables that fails passing the PrePARE tool.

An example is zg_Emon, but I found this at least in ta_Emon and wap_6hrPlev too.

$ PrePARE --table-path=/galileo/home/userexternal/pdavini0/post/jon-seddon-tables/Tables zg_Emon_EC-Earth3P-HR_control-1950_r1i1p1f1_gr_195001-195012.nc 

===========
The entry zg could not be found in CMOR table
===========

This actually make sense, since the variable in the table Emon is zg27, not zg.

        "zg27": {
            "frequency": "mon",
            "modeling_realm": "atmos",
            "standard_name": "geopotential_height",
            "units": "m",
            "cell_methods": "time: mean",
            "cell_measures": "area: areacella",
            "long_name": "Geopotential Height",
            "comment": "Geopotential is the sum of the specific gravitational potential energy relative to the geoid and the specific centripetal potential energy. Geopotential height is the geopotential divided by the standard acceleration due to gravity. It is numerically similar to the altitude (or geometric height) and not to the quantity with standard name height, which is relative to the surface.",
            "dimensions": "longitude latitude plev27 time",
            "out_name": "zg",
            "type": "real",
            "positive": "",
            "valid_min": "",
            "valid_max": "",
            "ok_min_mean_abs": "",
            "ok_max_mean_abs": ""

The problem is that the zg27 definition is still there in the most recent version of the tables (I mean, in the PCMDI version), so should we use a different name for those variables? Any opinions? Cheers, P

[goord]

It has out_name "zg" so I thought that is the field that PrePARE should look at, no?

[oloapinivad]

I suspect that PrePARE is looking at the variable name (i.e. the one written in the metadata) expecting zg27. Could it be that the out_name works only for the filename and not for the variable name?

[oloapinivad]

So here the story seems more complicated. I did a test trying to rename all the variable attributes and name in the zg_Emon file to zg27 in order to fit the table, so now it reads:

$ ncdump -h zg27_Emon_EC-Earth3P-HR_control-1950_r1i1p1f1_gr_195001-195012.nc | grep zg
netcdf zg27_Emon_EC-Earth3P-HR_control-1950_r1i1p1f1_gr_195001-195012 {
    float zg27(time, plev, j, i) ;
        zg27:standard_name = "geopotential_height" ;
        zg27:long_name = "Geopotential Height" ;
        zg27:comment = "Geopotential is the sum of the specific gravitational potential energy relative to the geoid and the specific centripetal potential energy. Geopotential height is the geopotential divided by the standard acceleration due to gravity. It is numerically similar to the altitude (or geometric height) and not to the quantity with standard name height, which is relative to the surface." ;
        zg27:units = "m" ;
        zg27:cell_methods = "time: mean" ;
        zg27:cell_measures = "area: areacella" ;
        zg27:history = "2018-09-24T11:56:03Z altered by CMOR: Reordered dimensions, original order: j i plev time." ;
        zg27:missing_value = 1.e+20f ;
        zg27:_FillValue = 1.e+20f ;
        zg27:coordinates = "latitude longitude" ;
        :variable_id = "zg27" ;

However, I still get the "variable zg could not be found in file". It's unclear to me where PrePARE is getting this information.

[goord]

Hi Paolo, I actually cmorized the 27-pressure level variables because Philippe had those levels in the output. The ncdump reads:

netcdf zg_Emon_EC-Earth3P-HR_hist-1950_r1i1p1f1_gr_197901-197901 {
dimensions:
    time = UNLIMITED ; // (1 currently)
    plev = 27 ;
    j = 512 ;
    i = 1024 ;
    bnds = 2 ;
    vertices = 4 ;
variables:
    double time(time) ;
        time:bounds = "time_bnds" ;
        time:units = "days since 1979-01-01 00:00:00" ;
        time:calendar = "proleptic_gregorian" ;
        time:axis = "T" ;
        time:long_name = "time" ;
        time:standard_name = "time" ;
    double time_bnds(time, bnds) ;
    double plev(plev) ;
        plev:units = "Pa" ;
        plev:axis = "Z" ;
        plev:positive = "down" ;
        plev:long_name = "pressure" ;
        plev:standard_name = "air_pressure" ;
    int j(j) ;
        j:units = "1" ;
        j:long_name = "cell index along second dimension" ;
    int i(i) ;
        i:units = "1" ;
        i:long_name = "cell index along first dimension" ;
    float latitude(j, i) ;
        latitude:units = "degrees_north" ;
        latitude:missing_value = 1.e+20f ;
        latitude:_FillValue = 1.e+20f ;
        latitude:bounds = "vertices_latitude" ;
    float longitude(j, i) ;
        longitude:units = "degrees_east" ;
        longitude:missing_value = 1.e+20f ;
        longitude:_FillValue = 1.e+20f ;
        longitude:bounds = "vertices_longitude" ;
    float vertices_latitude(j, i, vertices) ;
        vertices_latitude:units = "degrees_north" ;
        vertices_latitude:missing_value = 1.e+20f ;
        vertices_latitude:_FillValue = 1.e+20f ;
    float vertices_longitude(j, i, vertices) ;
        vertices_longitude:units = "degrees_east" ;
        vertices_longitude:missing_value = 1.e+20f ;
        vertices_longitude:_FillValue = 1.e+20f ;
    float zg(time, plev, j, i) ;
        zg:standard_name = "geopotential_height" ;
        zg:long_name = "Geopotential Height" ;
        zg:comment = "Geopotential is the sum of the specific gravitational potential energy relative to the geoid and the specific centripetal potential energy. Geopotential height is the geopotential divided by the standard acceleration due to gravity. It is numerically similar to the altitude (or geometric height) and not to the quantity with standard name height, which is relative to the surface." ;
        zg:units = "m" ;
        zg:cell_methods = "time: mean" ;
        zg:cell_measures = "area: areacella" ;
        zg:history = "2018-10-04T09:15:32Z altered by CMOR: Reordered dimensions, original order: j i plev time." ;
        zg:missing_value = 1.e+20f ;
        zg:_FillValue = 1.e+20f ;
        zg:coordinates = "latitude longitude" ;

[goord]

So there is no mention of zg27 anywhere in the output. Perhaps PrePARE is just confused with the multiple zg variables in the table, it thinks there should be a 27-pressure level variable too although you have the priority-1 zg7h 1variable in there...

[oloapinivad]

But actually in the Emon table there is only one zg27, and not "pure" zg. Perhaps is a bug on the PrePARE side?

[goord]

Maybe contact the PCMDI people about this issue?

[oloapinivad]

Yes, I will open an issue reporting it, perhaps we misunderstood something

[jonseddon]

Hi Paolo,

A colleague mentioned PCMDI/cmor#409 to me. I downloaded the file from Dropbox and it passed for me:

$ PrePARE --table-path /home/users/jseddon/primavera/cmip6-cmor-tables/Tables /group_workspaces/jasmin2/primavera3/cache/jseddon/test/zg_Emon_EC-Earth3P-HR_control-1950_r1i1p2f1_gr_195001-195012.nc
Processing: /group_workspaces/jasmin2/primavera3/cache/jseddon/test/zg_Emon_EC-Earth3P-HR_control-1950_r1i1p2f1_gr_195001-195012.nc

*************************************************************************************
* This file is compliant with the CMIP6 specification and can be published in ESGF  *
*************************************************************************************

Are you using a recent version of PrePARE? I'm using 3.3.2.

[oloapinivad]

Hi Jon,

I am using the version included in ece2cmor3, which as long as I know is the 3.3.2 too. I am validating against the Primavera CMIP6 tables, could this be the problem?

[jonseddon]

I'm using the PRIMAVERA CMIP6 tables too. There shouldn't be a difference between CMIP6_Emon.json in the PRIMAVERA version and the CMIP6 version. The only changes are in CMIP6CV.json and the addition of the extra PRIMAVERA*.json files. It would be good to test against the the genuine tables from https://github.com/PCMDI/cmip6-cmor-tables though. You should also check that you have the latest version of the PRIMAVERA tables. data_specs_version in CMIP6_Emon.json should be 01.00.23.

[oloapinivad]

Yes it is the same data specs: very strange, perhaps I am doing something wrong here. @goord, would it be possible for you to test PrePARE on my file in order to see you if you have the same issue? thanks! https://www.dropbox.com/s/d66yf59eyb27sms/zg_Emon_EC-Earth3P-HR_control-1950_r1i1p2f1_gr_195001-195012.nc?dl=0

[oloapinivad]

@goord , did you find the time to check if your files pass the PrePARE check? Thanks a lot for your help!

[goord]

Hi Paolo, not yet, will do it asap

[treerink]

This issue seems to be mentioned in general (not PRIMAVERA only) now in #462.

@oloapinivad, @goord can we mark this one as duplicate of #462 and close #275?

[oloapinivad]

yes closing this!