semver used in CSV no longer valid

[errordeveloper]

In efbfd7fd1e353b308f690557d24208dbb6f68d6a a version suffix was introduced to ensure multiple builds of the bundle can be published for each Cilium version.

Based on some preliminary testing a version string like 1.9.6-09e7dca appears to pass basic semver regex tests, albeit it reprsents a prerelease. This worked until now, but most recent addition of 1.9.6 failed certification tests.

===== Test: operator-metadata-linting =====

Operator Courier version:
-------------------------

2.1.10 (https://github.com/operator-framework/operator-courier/releases/tag/v2.1.10)

Validation Warnings:
--------------------

"csv metadata.annotations.description not defined"
"csv metadata.annotations.containerImage not defined"
"csv metadata.annotations.createdAt not defined"
"csv metadata.annotations.certified not defined."
"csv metadata.annotations.description not defined.Without this field, the description displayed in the tiles of the UI will be a truncated version of spec.description."
"csv metadata.annotations.createdAt not defined.Without this field, the time stamp at which the operator was created will not be displayed in the UI."
"csv metadata.annotations.containerImage not defined.Without this field, the link to the operator image will not be displayed in the UI."

Validation Errors:
------------------

"spec.version 1.9.6-09e7dca is not a valid semver (example of a valid semver is: 1.0.12)"
"UI validation failed to verify that required fields for operatorhub.io are properly formatted."

stdout:
-------

stderr:
-------

WARNING: csv metadata.annotations.description not defined [1.9.6-09e7dca/cilium-olm.csv.yaml]
WARNING: csv metadata.annotations.containerImage not defined [1.9.6-09e7dca/cilium-olm.csv.yaml]
WARNING: csv metadata.annotations.createdAt not defined [1.9.6-09e7dca/cilium-olm.csv.yaml]
WARNING: csv metadata.annotations.certified not defined. [1.9.6-09e7dca/cilium-olm.csv.yaml]
WARNING: csv metadata.annotations.description not defined.Without this field, the description displayed in the tiles of the UI will be a truncated version of spec.description. [5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/package.yaml]
WARNING: csv metadata.annotations.createdAt not defined.Without this field, the time stamp at which the operator was created will not be displayed in the UI. [5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/package.yaml]
WARNING: csv metadata.annotations.containerImage not defined.Without this field, the link to the operator image will not be displayed in the UI. [5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/package.yaml]
ERROR: spec.version 1.9.6-09e7dca is not a valid semver (example of a valid semver is: 1.0.12) [5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/package.yaml]
ERROR: UI validation failed to verify that required fields for operatorhub.io are properly formatted. [5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/package.yaml]
Resulting bundle is invalid, input yaml is improperly defined.

return code:
------------

1

-------------------
Execution Reference:

-> /cvp/cvp-isv-operator-metadata-validation-test/certified-ospid-e31ac831-7e72-42bb-baf9-f392ef7ea622-1b5f4a3e-a722-432d-a4e9-6837dd807df5/5dc4cb4a-a92b-434d-b1f4-c7b8d63da4b6/

[errordeveloper]

After some investigation it was understood that the test are perfomed by operator-courier, it looks this tool had been out of date in the certification system and has been updated recently. It currenlty used Python semver package version 2.8.1, which appears to mistreate leading zeros (https://github.com/python-semver/python-semver/issues/37).

$ pip3 install semver==2.8.1 ; python3 -c 'import semver; print(semver.parse("1.9.6-09e7dca"))' 
Collecting semver==2.8.1
  Using cached semver-2.8.1-py2.py3-none-any.whl (5.1 kB)
Installing collected packages: semver
  Attempting uninstall: semver
    Found existing installation: semver 2.13.0
    Uninstalling semver-2.13.0:
      Successfully uninstalled semver-2.13.0
Successfully installed semver-2.8.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/ilya/Library/Local/Homebrew/lib/python3.9/site-packages/semver.py", line 65, in parse
    raise ValueError('%s is not valid SemVer string' % version)
ValueError: 1.9.6-09e7dca is not valid SemVer string

However, newever versions work don't fail this test:

$ pip3 install semver==2.13.0 ; python3 -c 'import semver; print(semver.parse("1.9.6-09e7dca"))' 
Requirement already satisfied: semver==2.13.0 in /Users/ilya/Library/Local/Homebrew/lib/python3.9/site-packages (2.13.0)
OrderedDict([('major', 1), ('minor', 9), ('patch', 6), ('prerelease', '09e7dca'), ('build', None)])

Fixing operator-courier maybe possible, however it's question whether any changes to upstream project will propagate to the certification system soon enough.

It turns out that semver also allows +, and that's actually more meaningful for this use-case as it doesn't imply the notion of a prerelease.

This works well with either version of semver package in Python:

$ pip3 install semver==2.13.0 ; python3 -c 'import semver; print(semver.parse("1.9.6+09e7dca"))'
Collecting semver==2.13.0
  Using cached semver-2.13.0-py2.py3-none-any.whl (12 kB)
Installing collected packages: semver
  Attempting uninstall: semver
    Found existing installation: semver 2.8.1
    Uninstalling semver-2.8.1:
      Successfully uninstalled semver-2.8.1
Successfully installed semver-2.13.0
OrderedDict([('major', 1), ('minor', 9), ('patch', 6), ('prerelease', None), ('build', '09e7dca')])
$ pip3 install semver==2.8.1 ; python3 -c 'import semver; print(semver.parse("1.9.6+09e7dca"))' 
Collecting semver==2.8.1
  Using cached semver-2.8.1-py2.py3-none-any.whl (5.1 kB)
Installing collected packages: semver
  Attempting uninstall: semver
    Found existing installation: semver 2.13.0
    Uninstalling semver-2.13.0:
      Successfully uninstalled semver-2.13.0
Successfully installed semver-2.8.1
{'major': 1, 'minor': 9, 'patch': 6, 'prerelease': None, 'build': '09e7dca'}