Document (and possible change) the GEMINI versioning/release rules

PeterParslow commented 3 years ago

In this post, I will use the language adopted in OGC Policy Directive 18 (https://portal.ogc.org/public_ogc/directives/directives.php): major, minor, corrigendum (bug-fix). As in most version numbering, a minor release is "backwards compatible"; however "Bug-fix revisions may not be backward compatible, and in general will not be since there was judged to be an error in the previous version".

Up to GEMINI 2.2, GEMINI was released as a single PDF file, with simple versioning: any change implied a new version. The working group presumably made a judgement call on whether that was a major or minor version (1, 2, 2.1, 2.2). Note: at this stage, GEMINI did not include any encoding.

With 2.3 moving to online publishing, we were able to fix minor defects in a corrigendum; we chose not to release more than one a month, so these were "monthly releases", so we have GEMINI 2.3 July 2018, August 2018, September 2018, October 2018 - then, with most of the editorial stuff ironed out, it dropped to two in 2019 (January, March) and two in 2020 (January, April).

But we never documented how the decisions were made, and in fact we slipped in a few extra optional elements - with the reasoning that they were always allowed anyway (because they exist in ISO 19115/19139) and we were simply introducing guidelines so that they be used consistently.

In any versioning policy, there is always the judgement call of whether a "breaking bug fix" should be marked as some larger change. In effect, the working group will always have to make a judgement call. But it would be useful to publish the guidelines that the working group uses.

These may need to differ between the "logical" and encoding levels of GEMINI. Actual implementations are in practice all using the same encoding, so that "non-breaking" is currently a call at that level. But if/as we introduce alternative encodings, that will be more complicated.

Do please suggest some guidelines that we could use

nmtoken commented 3 years ago

I note in OGC Policy Directive 19 (on XML compatibility) that:

A bug fix revision of an XML implementation SHOULD NOT add new XML attributes or elements.

Which should guide us moving forward, but note too that in the same directive

NOTE 3 GML 3.2 uses a different namespace than GML 3.1 for all components. Using the test described in this sub-clause GML 3.2 is not backward compatible with GML 3.1. Thus, notwithstanding the version number, GML 3.2 is a major revision compared to GML 3.1. The error in designation pre-dates the clarification of OGC Standards Best Practice described in this document.

So what happened in the past stays in the past, and won't be reversioned

PeterParslow commented 3 years ago

I agree it shouldn't be reversioned.

But I would also say that given the open nature (built in extensibility) of GEMINI it's not clear to me that any of the monthly releases did add new XML attributes or elements. e.g. when we added Metadata standard name to GEMINI we merely provided guidance to improve consistency between what implementers were already doing. They were already free to use those XML elements.

"UK GEMINI2 defines a core set" "Additional metadata elements may be included in a metadata implementation. These elements should be taken from ISO 19115 Geographic information - Metadata" (https://www.agi.org.uk/agi-groups/standards-committee/uk-gemini/40-gemini/1049-metadata-guidelines-for-geospatial-data-resources-part-2).

nmtoken commented 3 years ago

It's not clear to me either what should happen with optional elements/attributes and versioning. As you say they could have been used prior to us describing them as having some purpose under GEMINI. We have some versioning with dates, and I feel that we should probably switch to versioning these with the same scheme as used for schematron. It seems a little odd at first to have say GEMINI 2.3.1 which is a bug fix change over 2.3.0 where the change could be in the documentation, or the schematron, or ... but it seems better than having some combination of date versioning, main schematron versioning, supplemental schematron versioning ... and then having to have some mapping so you know how the versioned components go together.

For schematron, I have a new rule, as below, which introduces a new requirement over the published GEMINI 2.3 schematron. I had thought of this as a bug fix, because it addresses a requirement that should have been included in the GEMINI 2.3 schematron (the requirement comes from current INSPIRE metadata TG).

<!-- ========================================================================================== -->
  <!-- Metadata Item 18 - Spatial Resolution                                                      -->
  <!-- Now inc. metadata/2.0/req/datasets-and-series/spatial-resolution                           -->
  <!-- ========================================================================================== -->
  <sch:pattern fpi="Gemini2-mi18">
    <sch:title>Spatial Resolution</sch:title>
  </sch:pattern>
  <sch:pattern is-a="TypeNillablePattern" id="Gemini2-mi18-Nillable">
    <sch:param name="context"
      value="//gmd:MD_Metadata[1]/gmd:identificationInfo[1]/*[1]/gmd:spatialResolution/*[1]/gmd:distance"
    />
  </sch:pattern>
  <sch:pattern fpi="Gemini2-mi18-resolution-and-scale">
    <sch:title>Spatial Resolution resolution-and-scale</sch:title>
    <sch:p>We need to test as per INSPIRE TG Requirement 1.5 that for a dataset or dataset series that we have EITHER equivalent scale or a resolution distance WHERE they are described, but NEVER both.</sch:p>
    <sch:let name="srRD" value="count(//gmd:MD_Metadata[1]/gmd:identificationInfo[1]/*[1]/gmd:spatialResolution/*[1]/gmd:distance/gco:Distance)"/>
    <sch:let name="srES" value="count(//gmd:MD_Metadata[1]/gmd:identificationInfo[1]/*[1]/gmd:spatialResolution/*[1]/gmd:equivalentScale/gmd:MD_RepresentativeFraction/gmd:denominator/gco:Integer)"/>
    <sch:rule context="//gmd:MD_Metadata[1]/gmd:identificationInfo[1]/gmd:MD_DataIdentification">
      <sch:report
        test="($hierarchyLevelCLValue = 'dataset' or $hierarchyLevelCLValue = 'series') and ($srES &gt; 0 and $srRD &gt; 0)"
        > MI-18a (Spatial Resolution): Spatial resolution for data set or data set series shall be given using either equivalent scale or a resolution distance, provided that these have been specified for the described data sets. If both ways have been specified, only one of the ways shall be used.
        We have equivalent scale <sch:value-of select="$srES"/> and resolution distance <sch:value-of select="$srRD"/>
      </sch:report>
    </sch:rule>
  </sch:pattern>

Looking at it using OGC Policy Directives 18/19, it doesn't add new XML attributes or elements rather it restricts them, so we could use bug fix numbering, giving us 2.3.1

PeterParslow commented 3 years ago

We may need separate numbering for different "parts":

Major.minor.corrigendum for documentation (especially "guidance" parts?) Major.minor.bugfix for (XML) encoding Major.minor.bugfix for Schematron - in the same way that W3C & the OGC distinguishes between the version numbers of the XSD files themselves as distinct from major.minor versioning of the schema (identified by the namespace).

So in this example, the Schematron number would progress (e.g. to 2.3.1) whilst still being a Schematron designed to check the encoding that's documented at present in "2.3 April 2020".

However, we also considered release numbering, so that GEMINI 2.3.1 or 2.4 would be a bundle of documentation (guidance & logical level rules), encoding & schematron - where each of those should contain a compatible set of changes at the date of release. We could then stick to release numbering, with individual "draft" (i.e. not released) fixes bundled into a release.

For this example, I agree with you that it's enforcing a rule that was always intended (and is hinted at by a comment on GEMINI Equivalent scale). It isn't strictly there in 19115 or INSPIRE, because although each 19115 spatialResolution can only be one or the other, 19115 does allow more than one spatialResolution to be given. (19115-1:2014 says that the reason is to allow expressing a min/max range of resolutions, not that that's strictly relevant here).

PeterParslow commented 3 years ago

I think we also need to allow some issues to be resolved independent of a release. I am thinking of ones that affect the "ancillary pages" in ways that should not impact on implementation, or are "bug fixes" to those pages making them say more clearly what they were intended to say.

Examples could include: #23 ; this issue!; #24; #22 ; #5 - perhaps a number of others

6footdestiny commented 3 years ago

It makes sens e to separate 'code' changes and 'support docs' for versioning purposes - they have their own logic and audiences. The code versioning is more important IMO as it may have machine relevance to some software implementers. The non-code should properly be versioned but at a less granular fashion.

PeterParslow commented 3 years ago

2021-06-08 Principles:

Clarifying documentation (e.g. the "home page") doesn't need to be part of a release. Examples, #2, #24, #31
Clarifying documentation even on the element page could be the same: decide on a case by case basis, e.g. #8, #10, #14
Any change that requires changes to implementing software should be bundled into a release. This would include e.g. #15, #25 - with a new third part of the version number
A breaking change of the way it has to implemented has to be bundled into a release, with a new second (or first) part of the version number.

Question: does the Metadata standard version need to carry all three parts?

Before we make a release, we need to ensure that 2.3 is still accessible, even as a ".old" in this repository.

Numbering rules:

Move to three part release numbering.
Retain the "last updated" date on each of the pages.
When changing an element, need to add a summary of the change as well as the date.

PeterParslow commented 3 years ago

Introduce "release" and "immediate" flags to distinguish fixed issues.

Next "bundled release" should be 2.3.1 (or 2.4), to avoid confusing people!

PeterParslow commented 3 years ago

I've introduced a new column to the project ("With AGI to publish") to separate those:

pending (with us/Peter) release (to AGI)
with AGI (technically, BlueStar) to publish
pending closure e.g. published or no change expected but the WG hasn't looked yet,

archaeogeek commented 1 year ago

agiorguk / gemini

Document (and possible change) the GEMINI versioning/release rules #27