Expand CI trigger-branches to include multiple develops and unstables

[ajnelson-nist]

Background

UCO and CASE currently use a testing workflow that reviews proposals' Pull Requests before merging for their respective next releases. The workflow includes downloadable "Nightly" build artifacts. This testing process is documented here.

With the 1.0.0 releases of UCO and CASE, the ontologies now have two "Prerelease" tracks:

• Backwards-compatible proposals fit for the next minor release, and
• Backwards-incompatible proposals fit for the next major release.

Following the Git Flow model, we stage the changes for the next release in develop. That model does not prescribe an exact spelling for the "develop that stages stuff for the next major release," so we need to pick a branch name. Ideally, that branch name will have a pattern that also suits:

• the unstable testing branches on the Archive Git repositories (UCO's, CASE's),
• the downloadable prerelease ontology builds that CASE provides, and
• the pre-computed validation results used in CASE-Examples and the CASE website.

This proposal is proposed as a fast-track proposal, lacking ontological or modeling impact, and instead being about revising the development and testing infrastructure. The impact is more scoped to the CDO, UCO, and/or CASE role-equivalence of CASE's Product Manager. From the Ontology Committees, feedback is requested on branch naming schemes, and on whether anything should be added to the CONTRIBUTE.md file or its linked resources. The proposal is also going before the Ontology Committees so we may see an illustration of how a proposal can be drawn across a minor and major release.

Fast-track proposals require a low amount of risk to the ontology and its adopters, and the submitter believes this proposal is acceptably low risk because a reactive, corrective action would be effectively scoped within the ontology to changing Git branch names.

This proposal should be considered scoped to CASE as well as UCO.

Requirements

Requirement 1

UCO needs a branch to serve as a merge target for Pull Requests that are part of the next major release (and thus could wait for possibly a year before merging into develop).

Requirement 2

UCO's Continuous Integration (CI) needs to trigger based on the branch names for:

• Current develop (which it already does).
• The "develop" staging for the next major release.

Requirement 3

This is a request from the current testing-process maintainer, rather than a requirement.

UCO's CI needs to trigger based on the branch names for:

• Current unstable (which only happens in the UCO-Archive repository. (And likewise for CASE.)
• The unstable branch for the next major release.

Requirement 4

This is a less certain requirement to the submitter, but the OC should consider it.

The name scheme selected should be able to distinguish between the next major release and the following major release.

Risk / Benefit analysis

Benefits

• Requirements 1 and 2 enable UCO's ability to track compatibility of accepted next-major features over the course of months. Looking at the first figure on the Git Flow page, the leftmost branch "Feature for future release" can be the "Next major develop" branch, and incorporate "Catch-up" merges from the current develop branch to continually review compatibility.
• Requirement(/Request) 3 eases a setup step that needs to happen the UCO-Archive repository each time the unstable branch is reset to develop. The first commit there generally needs to be enabling CI for the unstable branch. A step would thus be saved for the testing maintainer if UCO's main branches included having unstable be a triggering branch, even if the UCO repository itself doesn't maintain an unstable branch.
• Requirement 4 supports use cases where the OC might decide on a gradual deprecation on some class or property usage, and need to give adopters the interval of a major release to revise their implementations and/or graphs.

Risks

• A branch name scheme may prove problematic for some unconsidered reason. However, the impact of changing a name scheme is unlikely to expand past merge notes in the Git history.
• Updates made for the next major release(s) will need to follow along with the current releases, which implies extra work for the Product Manager. "Follow along" can be an extra step in the Pull Request review, or it can be deferred.

Competencies demonstrated

Competency 1

A SHACL class-based constraint in UCO 1.0.0 is proposed for constraining to a narrower subclass. For illustration, UCO 1.0.0's observable:ApplicationAccountFacet contains this shape (excerpted):

observable:ApplicationAccountFacet
    a sh:NodeShape ;
    sh:property [
        sh:class observable:ObservableObject ;
        sh:maxCount "1"^^xsd:integer ;
        sh:nodeKind sh:IRI ;
        sh:path observable:application ;
    ] ;
    sh:targetClass observable:ApplicationAccountFacet ;
    .

Suppose the ontology committee accepts for the next-major release that this should constrain to observable:Application rather than the generic observable:ObservableObject. This can be the implementation for the next-major branch ...

observable:ApplicationAccountFacet
    a sh:NodeShape ;
    sh:property [
        sh:class observable:Application ;  # <-- Changed
        sh:maxCount "1"^^xsd:integer ;
        sh:nodeKind sh:IRI ;
        sh:path observable:application ;
    ] ;
    sh:targetClass observable:ApplicationAccountFacet ;
    .

... and this can be the implementation for develop, for the next minor release:

observable:ApplicationAccountFacet
    a sh:NodeShape ;
    sh:property
        [
            # This PropertyShape is unchanged.
            sh:class observable:ObservableObject ;
            sh:maxCount "1"^^xsd:integer ;
            sh:nodeKind sh:IRI ;
            sh:path observable:application ;
        ] ,
        [
            # This PropertyShape will be dropped in the next major release.
            # The differing constraint is this sh:class line:
            sh:class observable:Application ;
            # The warning in place until the next major release is a message accompanying a sh:Warning severity.
            sh:message "Deprecation notice: the object referenced by observable:application will be required to be an observable:Application in UCO 2.0.0, instead of a generic observable:ObservableObject."@en ;
            sh:severity sh:Warning ;
            sh:path observable:application ;
        ]
        ;
    sh:targetClass observable:ApplicationAccountFacet ;
    .

Competency Question 1.1

What is an example process for deprecation of genericity of a constraint, i.e. implementing a backwards-incompatible constraint-tightening?

Result 1.1

This process could be followed to implement a backwards-incompatible change with a warning of it coming:

1. Implement the backwards-incompatible constraint changes in a separate PropertyShape blank node, with a severity to the new shape less severe than the SHACL default sh:Violation.
2. Merge the new shape into develop.
3. Remove the deprecation message and warning severity, and move the new constraints into the prior PropertyShape.
4. Merge the updated shape into develop-2.0.0 (or whatever the "next major" develop branch is).

This process can all be handled on the same day after a single Solutions Approval vote from the OC.

Competency Question 1.2

What is an example process for a deprecation that will play out over two major releases?

Result 1.2

This process is as with result 1.1, except:

• Step 1's sh:severity can be sh:Info.
• The merge into develop-2.0.0 (next-major) can just upgrade the severity to sh:Warning, before step 3.
• Steps 3 and 4 can merge into develop-3.0.0 (next-next-major).

Solution suggestion

These solution points should apply to CASE as well as UCO. Also, references to 2.0.0, 3.0.0, etc., should follow that pattern for future major releases.

• Revise the Github Actions in UCO to trigger on develop* and unstable* where they currently trigger on develop.
• Define a Git branch develop-2.0.0. Use this branch as the target for all Pull Requests incompatible with any 1.x.0.
• When a change is accepted that the OC feels needs two major releases to deploy, define a Git branch develop-3.0.0.
• At some semi-regular interval, or as part of closing Pull Requests, merge develop into develop-2.0.0.
• When the last minor release of the 1.x.0 series is finished, merge develop-2.0.0 into develop.

The below parts of the suggested solution are not necessarily the responsibility of the Ontology Committee, but are an illustration of process impact.

• For future-unstable branch management:
  • Define unstable-2.0.0. Use this to test all Pull Requests targeting develop-2.0.0.
  • Merge unstable into unstable-2.0.0 as part of testing Pull Requests targeting develop.
  • Archive and stop generating unstable-2.0.0 after the last minor release of the 1.x.0 series.
• For CASE's nightly builds:
  • Add CASE-develop-2.0.0.ttl as the nightly build for the next major release. Remove this after the last minor release of the 1.x.0 series.
  • And likewise for the unstable nightly builds.
• For precomputed validation results (.ttl files) in CASE-Examples (e.g. Oresteia) and the CASE website (e.g. Owl Trafficking):
  • Where there is a file ${example}_validation-develop.ttl, generate ${example}_validation-develop-2.0.0.ttl Remove this after the last minor release of the 1.x.0 series.
  • And likewise for the unstable results.

Coordination

• Tracking in Jira ticket OC-276
• [x] Administrative review completed, proposal announced to Ontology Committees (OCs) on 2022-10-26
• [x] Solution announced to OCs on 2022-10-26
• [x] Solutions Approval to be discussed in OC meeting, 2022-11-17
• [x] Solutions Approval vote occurred, passing, on 2022-11-17
• [x] Solutions development phase completed.
• [x] UCO Implementation merged into develop
• [x] CASE Implementation merged into CASE develop
• [x] Milestone linked
• [x] Documentation logged in pending UCO release page
• [x] Documentation logged in pending CASE release page

[ajnelson-nist]

Depending on the result of the committee decision on removing concept IRIs, our first test of the develop-2.0.0 branch could come with Issue 491.

[ajnelson-nist]

Depending on the result of the committee decision on removing concept IRIs, our first test of the develop-2.0.0 branch could come with Issue 491.

Issue 501 would also be impacted by the deprecation plan.

[ajnelson-nist]

Fodder for the deprecation plan, which I acknowledge is a bit of scope-creep of this issue about making multiple develop branches:

In the early discussions for our ontology versioning strategy, a working group had concluded we should retain concepts for a long period, e.g. 10 years.

If we choose to retain deprecated concepts instead of delete them once a major version rolls around, I suggest we use OWL designations for concept deprecations, in addition to the SHACL warnings.

E.g. in Issue 501, we're looking at deprecating a property. If we choose to retain that property IRI for a long period or indefinitely, it should pick up this designation:

observable:storageCapacityInBytes a owl:DeprecatedProperty .

A SHACL shape can be defined, independent of any UCO class, that enforces the concept not be used:

observable:storageCapacityInBytes-deprecation-shape
    a sh:NodeShape ;
    sh:property [
        a sh:PropertyShape ;
        sh:maxCount 0 ;
        sh:message "The property observable:storageCapacityInBytes is deprecated."@en ;
        sh:path observable:storageCapacityInBytes ;
    ] ;
    sh:targetSubjectsOf observable:storageCapacityInBytes ;
    .