Mapping between standard names and GRIB codes

[atmodatcode]

Title

Three not resolving hyperlinks in last sentence of section 3.3.

Detailed Proposal

I would like to request that the following three non-resolving hyperlinks are corrected.

"Here are lists of equivalences between the CF standard names and the standard names from the [ECMWF GRIB tables]'(http://cf-pcmdi.llnl.gov/documents/cf-standard-names/ecmwf-grib-mapping), the [NCEP GRIB tables]'(http://cf-pcmdi.llnl.gov/documents/cf-standard-names/ncep-grib-code-cf-standard-name-mapping), and the [PCMDI tables]'(http://cf-pcmdi.llnl.gov/documents/cf-standard-names/pcmdi-name-cf-standard-name-mapping)."

Thanks a lot. Regards Angelika

---

12 May 2022. Title changed to reflect the discussion. 29 Aug 2023. Pull request https://github.com/cf-convention/cf-conventions/pull/452 to delete the line concerned. 20 Oct 2024. New PR #552.

[taylor13]

Doing a quick search of our (PCMDI's) latest website, I haven't found these pages (yet). Doing a web search, I came across a 2014 presentation by @japamment discussing equivalences between grib variables and CF variables. I wonder if anything written came out of that discussion that we could link to?

We might also generate a list of CMIP variable names associated with each standard name and link to that. Or we could simply eliminate the sentences given above. Votes?

[durack1]

There are archived versions of these tables from 15th February 2013 at:

ecmwf-grib-mapping ncep-grib-code-cf-standard-name-mapping pcmdi-name-cf-standard-name-mapping

[larsbarring]

I suggest that the sentences are removed altogether (the links Paul gives go to the web archive). In particular references to GRIB tables may be outdated. An up-to-date cross-reference table between standard names and CMIP variables seems like it could be useful, but is maybe better placed over at the CMIP website, in which case it could be cited in this section, and possibly elsewhere (as in the standard name table).

[durack1]

If an intention of mapping the CMIP6 variable names with CF standard_name entries, that would be relatively trivial, if you consider the general example tas thanks to the work of @martinjuckes and his CMIP6 Data Request (v01.00.33, see also Data_Request_Home)

[taylor13]

If the grib tables are reasonably up to date, I think we should retain links to them; it would help some users correctly identify data of interest outside of CF.

[taylor13]

Jeff Painter tells me that "Standard names have been moved to Github along with the CF Conventions document and most of what it references. See https://cfconventions.org/standard-names.html ." Is that of any help?

[JonathanGregory]

Dear all

I think that the mapping of standard names plus other CF metadata to other metadata conventions such as PCMDI and GRIB is an important task and a valuable resource. At the start of the CF convention, we put the correspondences into the standard name table to make standard names more attractive to potential users, since they were a new idea. This information was a kind of advertisement for the possible use of standard names.

However, 20 years later, standard names are now an established convention. I tend to agree with Lars that these sentence should be removed from the conventions. I also think that the <grib> and <amip> tags should be removed from the standard name table. I think that the equivalences should be kept in other files, which aren't part of the CF convention itself. They belong equally to the other conventions involved, and we have never made an effort to keep the up-to-date. However, we can keep tables of equivalence on the CF website, or make links from the CF website to other websites if someone else is maintaining them.

Best wishes

Jonathan

[larsbarring]

I agree that such metadata mappings are important, but they need to be kept up-to-date and/or versioned. Linking to outdated information are likely to cause more trouble than being helpful. Regarding the <grib> and <amip> columns it the standard name table like to idea to move the information into separate tables ( I think that this has been discussed before but I cannot find where/when). Moreover, in the <grib> column there are sometimes Exxx or Nyyy (xxx, yyy being numbers), and it not immediately clear what this is; I guess it refers to ECMWF or NCEP local grib tables. All in all, I think that might be no small effort to collect such a mapping, and to keep it up-to-date. Maybe there are already activities in this direction elsewhere?

[durack1]

The (WMO?) GRIB1 and GRIB2 codes, along with the ECMWF mappings for netcdf can be viewed at https://apps.ecmwf.int/codes/grib/param-db/?filter=netcdf - interesting to see how different things are, so for "sea ice area fraction" we have ECMWF:

Name	Short/CMIP Name	Units	Parameter ID	GRIB1	GRIB2	NetCDF
Sea ice area fraction	ci	(0 - 1)	31	y	y	y

And CF/CMIP6 (CF Standard Name Table v79, 19 Mar 2022 - Canonical Units: 1; AMIP name: sic; GRIB code: 91):

Name	Short/CMIP Name	Units	CF standard_name
Sea Ice Area Fraction	siconc	%	sea_ice_area_fraction

[feggleton]

I would agree to remove the GRIB and AMIP columns from the standard name table as there are not being maintained.

[JonathanGregory]

@larsbarring wrote

Moreover, in the <grib> column there are sometimes Exxx or Nyyy (xxx, yyy being numbers), and it not immediately clear what this is; I guess it refers to ECMWF or NCEP local grib tables.

That's right. I put them in the prototype table more than 20 years ago (if my memory is correct) and I don't think they have been revised.

[taylor13]

I think the equivalences are useful, but it appears that no one has volunteered to keep them up-to-date. So, reluctantly, I vote to remove the unmaintained columns from the table. Perhaps at some point, some one will stand-up and maintain this information elsewhere and we can link to it.

[japamment]

I agree that the GRIB and AMIP columns should be removed from the standard name table as the mappings have not been maintained in a long time. The non-resolving hyperlinks should also be removed from the conventions document.

@durack1 is correct that equivalences between standard names and CMIP6 variable names can be found from the CMIP6 Data Request documentation: https://clipc-services.ceda.ac.uk/dreq/mipVars.html. We could add that link to the conventions document and the standard names/vocabularies web page.

Mapping standard names to other vocabularies, not only GRIB and CMIP, is an important issue and was the subject of a hackathon held during the CF 2022 workshop (co-convened by @gwemon, @alko-k and myself). We had some presentations as part of the session, available from the meeting Google drive. Following on from the hackathon, a CF working group is being set up to progress the mapping of standard names to the interoperability framework resulting from the RDA I-ADOPT group: https://doi.org/10.15497/RDA00071. Anyone with an interest is welcome to join the group (email alison.pamment at ncas.ac.uk).

The reason for wanting to map standard names to I-ADOPT is that it is a generic framework that can provide a bridge to many other vocabularies, rather than mapping between specific pairs of vocabularies such as CF and GRIB. I-ADOPT is fairly new (published in May 2022) and there are some early implementations, for example, OGC have added it to their SensorThings API (used mostly for instrument observations). The I-ADOPT ontology has been added to the NERC Vocabulary Server (NVS), where CF standard names are also served as http://vocab.nerc.ac.uk/collection/P07/current/. NVS implements all the technology needed to serve mappings and is a far more powerful solution than listing mappings as part of the standard name table on the CF website; I think this is the route we should be looking to for the future.

I plan to add information to the vocabularies web page to give more prominence to the P07 route for accessing standard names (I'd like CF users to understand that it is a completely 'official' version). As the I-ADOPT work progresses I will be able to add more information regarding accessing mappings.

I'd like to reassure the CF community that mapping to I-ADOPT in no way affects the usual CF processes for adding standard names, it is complementary to our existing way of working.

Best wishes, Alison

[martinjuckes]

I'll also vote to remove them.

The GRIB community could be seen as a user community: we can support them if they want advice on mappings, but they should be responsible for maintaining the information about what their variables mean. GRIB is a bit like the data request in that it is designed to deal with a bundle of specific user and application requirements.

[JonathanGregory]

Dear all

Angelika @atmodatcode opened this issue last year as a defect, because of broken links. The subsequent discussion converged on a decision to remove the GRIB and PCMDI equivalences from the standard name table. That's a larger change than resolving a defect. However, the defect remains. As a limited measure, I have drafted pull request #452 to delete the line with the broken links, which refer to obsolete tables of equivalences, and thus fix the defect. (Earlier in this discussion, @durack1 gave links to archive versions of these tables, for the record.)

I propose that this PR be merged immediately, since a long time has passed with no disagreement and plenty of support was expressed for the decision to remove the equivalences. Please could someone check and merge it?

After that, we can keep this issue open as an enhancement rather than a defect, for the larger tasks of revising the standard name table format (defined by Appendix B) and its contents. We all agreed that it would be good to have equivalences maintained elsewhere.

Best wishes

Jonathan

[larsbarring]

Thanks @JonathanGregory, I have just merged the PR. To keep the issue open to take care of the remaining parts sound good.

[JonathanGregory]

Thanks, @larsbarring. I've relabelled this issue as an enhancement.

[JonathanGregory]

Dear Alison @japamment, Fran @feggleton, Ellie @efisher008

In this issue we have agreed already to remove the AMIP and GRIB equivalences from the standard name table. Could you schedule this change in some near-future version of the table?

At the same time, we need to modify Appendix B to remove the relevant parts of the xml. I have prepared #502 with that effect. Please could you check it? We shouldn't merge it until the table itself is modified, I suggest.

Best wishes

Jonathan

[larsbarring]

This issue specifically deals with the GRIB and AMIP columns/entries of the xml files. In this respect I support the changes suggested by @JonathanGregory in the associated PR.

But I also suggest that when we deal with the format of the standard name xml file specified in Appendix B we should also consider #500 and #132 before releasing the next version of the standard name table (to my mind these involve small and rather uncontroversial changes).

[larsbarring]

See cf-convention.github.io/#433 regarding the last sentence in my previous comment.

[japamment]

@larsbarring @JonathanGregory,

As indicated in my previous comment I support removal of the GRIB and AMIP columns from the xml and html versions of the standard name table.

To remove them from the xml file requires modification of the xsd schema, and correct rendering into the html version will also require modification of the xsl file (contained in the xsl/html directory associated with each standard name table version) which is used by the xsltproc command. Both files will need to be changed at the same time and we will need to test the process of creating the html from the xml to make sure it works as expected.

The CEDA vocabulary editor will need to be updated to no longer output the amip and grib fields. As a temporary measure until the editor is updated, we can apply a post-processing script to the xml file to achieve the same result. However, this step should not be carried out until we are happy with the xsd/xsl file changes.

[larsbarring]

@japamment

I absolutely agree with you. Updating the xsd and xsl schema and stylesheet files is part of the changes summarised in https://github.com/cf-convention/cf-convention.github.io/issues/457. It is deliberately not touching what is relevant for this issue. Rather, I see that effort as an way to pave the way for implementing the change suggested here, except for what concerns the CEDA vocabulary editor.

[JonathanGregory]

I have created PR #552 to delete the description of the AMIP and GRIB tags in the standard name table format definition of Appendix B. This is actually the same contents as PR https://github.com/cf-convention/cf-conventions/pull/502, which wasn't merged and is now broken. Since we've agreed to make this change to the standard name table, and I expect that it will be done before CF 1.13, it would be good to alter Appendix B in CF 1.12, I think. Does anyone support or disagree with this suggestion?

[durack1]

@JonathanGregory since this discussion started a couple of years ago, we've been moving increasingly toward a variable registry for MIPs that will prominently have the standard_name identifier available for matching. Not that it should change the plans to remove the deprecated names etc immediately, but if the appetite to keep the CMIP connection alive, this will be far more easy in the future as the MIP variable registry is realized

[japamment]

Dear Jonathan @JonathanGregory @larsbarring

I support the changes in PR #522 and agree that this is the right time to remove mention of the GRIB and AMIP tags from Appendix B of the conventions. The standard names editor software is being worked on now at CEDA and the updated version will implement all the recently agreed changes to the XML file format:

• Additional header tags for CF-conventions string and first published date #500 #511
• No GRIB or AMIP tags
• Allow alias id tags to contain more than one standard name entry_id reference #509

In recent standard names updates we have implemented the changes agreed in #500, #509 and #511 by hand editing the XML, so having these now automated in the editor is a significant improvement. The new software version will be deployed in time for the next update of the standard name table which is planned for 11th/12th November 2024.

Best wishes, Alison

[JonathanGregory]

Dear Alison @japamment

Thanks for checking this, and for the news of progress with the editor. That's all very good. Since this proposal was made a long time ago, no concerns have been raised, and enough support has been expressed, I believe we don't need to wait for another three-week period. I will therefore merge it now.

Best wishes

Jonathan