Add git tags for deterministic builds/tests of trick + koviz operations

[cazlo]

What

Release tags periodically

Why

There is an upstream dependency in nasa/trick on nasa/koviz for trickops tests it performs in CI. See also trickops CI definition.

Currently this upstream dependency is "open", pointing to the HEAD of master. Open dependencies bring with them several operational challenges including:

• Security Risks: Open dependencies might contain vulnerabilities that can be exploited if not regularly updated or monitored.
• Introduction of Fault: If dependencies are not properly locked to specific versions, new updates could break builds or introduce new issues in the CI pipeline.

To support deterministic, stable CI test operations, open dependencies are usually replaced with references to a specific version of the software, e.g. a specific git sha, a git tag, etc.

This is related to several standards including:

NASA 7150.2D (NASA Software Engineering Handbook)

• SWE-081 - The project manager shall identify the software configuration items (e.g., software records, code, data, tools, models, scripts) and their versions to be controlled for the project.

NIST 800-218 (Secure Software Development Framework - SSDF)

• PO.5: Archive and Protect Software Development Artifacts
• PO.3.2: Reproducible Builds

NIST 800-53 (Security and Privacy Controls for Information Systems and Organizations)

• SI-2: Flaw Remediation
• SA-10: Developer Configuration Management
• SA-12: Supply Chain Protection

Recommended Process Implementation

1. some feature branch for nasa/koviz gets merged into master
2. branch in nasa/trick where we update CI to point to nasa/koviz @ master commit from step 1
3. assert feature branch of nasa/trick has all passing tests
4. create new tag in nasa/koviz from the commit to master in step 1
5. update feature branch in nasa/trick to point to nasa/koviz @ the tag in step 4

Recommend additionally to use calver (e.g. v2024.09) or semver (e.g. v1.2.3) for the tag.

[cazlo]

An alternative solution which could be utilized to achieve the same desired end state with potentially less ongoing labor expenditure necessary:

What

Absorb nasa/koviz source code into nasa/trick using a utility which preserves its git history.

Why

There appears to be tight coupling between nasa/trick and nasa/koviz; trick needs koviz to test and koviz needs trick to get data to visualize. To provide data around this statement, in analysis of the NASA org, I am finding only reference to this code in nasa/trick: https://github.com/search?q=org%3Anasa%20github.com%2Fnasa%2Fkoviz&type=code

• docs/howto_guides/Realtime-Best-Practices.md
• docs/documentation/miscellaneous_trick_tools/TrickOps.md
• .github/workflows/trickops.yml

Expanding the search further to all of public GitHub yields similar results from various forks of trick.

The tight coupling between the components introduces a potential circular dependency with respect to testing, if they are managed as 2 separate codebases/CI runtimes. This appears to be worked around now by the CI tests of nasa/koviz primarily being smoke-check type functions, as opposed to higher fidelity tests which include coverage analysis.

Merging the 2 codebases would replace the need to download and unzip the nasa/koviz artifact at nasa/trick CI runtime, potentially saving a ROM (rough order of magnitude) of between tens of seconds and minutes in the nasa/trick CI operations.

We could reuse the existing trick Release process, avoiding the need for the 5 manual steps described in the original feature request.

Having 1 git commit which describes the entire system, as opposed to 2, ensures deterministic, reliable CI operations. It helps to avoid situations where a change in nasa/koviz introduces fault into nasa/trick in a released version of nasa/trick.

How

1. In nasa/trick, create a feature branch from master
2. In the feature branch do something like git subtree add --prefix koviz https://github.com/nasa/koviz.git master
3. Manually merge together the nasa/trick/.github and nasa/trick/koviz/.github folders to unify the CI
4. Merge feature branch into master
5. Archive nasa/koviz

Risks

• It might be hard to show a unified test coverage report in CI.
• Increase nominal I/O for pulls of nasa/trick. The additional I/O comes from pulling the git history of nasa/koviz and the additional source code. Could potentially be mitigated by use of the --squash feature of git subtree.
• Other private uses of nasa/trick and/or nasa/koviz may be impacted.
• Links to nasa/koviz in Engineering documentation would need to be updated. To mitigate this, before we archive nasa/koviz as step number 5 of the procedure, we would probably want to update the README or add a banner or something to point people to nasa/trick/koviz.

[keithvetter]

Hi cazlo! I think this makes good sense, "To support deterministic, stable CI test operations, open dependencies are usually replaced with references to a specific version of the software, e.g. a specific git sha, a git tag, etc."

I haven't resisted tagging so much as waiting for the actual need and to make it meaningful. It's very hard to draw a line because the latest is really the greatest. A tag seems arbitrary. That said, I get that it'd be good, especially since koviz is used in other tool's CI process, to sit on a static koviz commit rather than always pointing to HEAD. That gives confidence that a koviz update didn't change the response - even if the response is better. I guess I've always thought that people would want the latest hash until a point where koviz got more established. Maybe this issue signifies that milestone. Koviz could be growing out of, "Oh, pull the latest."

What this would do as well is, draw a line where I could create a corresponding RPM. This issue is pertinant as well to conversations today where koviz is being included in various projects.

[keithvetter]

I'm closing this issue after tagging. I think the next tag will be 1.0.0 and will be customer driven.

[cazlo]

Awesome, thank you!

[keithvetter]

You're welcome!