SCS Standard Images reference YAML not clearly communicated

[markus-hentsch]

Putting myself into the perspective of a CSP that aims for SCS conformance I noticed the following shortcomings while reading the "SCS Standard Images" document in the official SCS docs at https://docs.scs.community/standards/scs-0104-v1-standard-images and trying to reach conformance to it.

Current shortcomings

The standard spends a lot of time elaborating on the process of how images are standardized and how the standardization is implemented using YAML descriptions but little on how to figure out the currently standardized images. My primary question as a CSP would be: "What images do I have to provide to be SCS compliant?". I don't feel this question is answered in a confident way. The standard currently only states this small sentence under the headlines "Lifecycle considerations"/"YAML lifecycle":

The YAML file is generally located in this repository under /Tests/iaas.

Problems with that:

• This should be some kind of clickable permalink. The "in this repository" is a bad reference when reading the docs.scs.community page as a docs page.
  • Even if discovering and clicking the "GitHub" link on the top right of the page, it forwards to the docs repository, not the standards repository where the file resides.
• In my opinion, the wording of the sentence and its location in the standard does a poor job at communicating that this the central important YAML to achieve conformance with the standard. It sounds more like an example file similar to the code blocks illustrated in the preceeding sections.
  • At the bottom of the standard there is a link to OSISM's image manager which also ships YAML templates for images^1 where as a CSP I could see myself asking instead: "are these the YAMLs I have to apply for the standard?"

Suggestions for improvement

In general, I would like to improve the discoverability of the images YAML file^2 and emphasize its role in relation to achieving standard conformance, just like in other standards (e.g. the flavor standard where mandatory and recommended flavors are listed directly in the standard):

• Provide a clickable reference link to the currently valid images YAML file^2 (that also works on the rendered docs page) OR if possible, directly render the YAML into the docs page.
• Better communicate that the referenced YAML file is the binding standard regarding the list of mandatory/recommended images that need to be provisioned to achieve conformance to the standard.

I'd like to hear your opinion on this.

[mbuechse]

This is not controversial – I agree on all accounts. Thanks for spotting this.

[mbuechse]

For context: When we crafted this standard, the docs page wasn't as advanced nor as widely adopted as today. So it should be updated without question.

[mbuechse]

On second thought, the crux here is the following: from my point of view, this standard merely regulates the format of the YAML file; which YAML file is ultimately used is not part of the standard, but a parameter of the certificate subject. Well, apparently, this doesn't seem to be the most practical point of view... So this turns out to be something we should discuss in the SIG after all(!)

[mbuechse]

Let me expand on my previous statement. Currently, what YAML is relevant is solely determined in the certificate scope:

      - name: Standard images
        url: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0104-v1-standard-images.md
        checks:
          - executable: ./iaas/standard-images/images-openstack.py
            args: -c {os_cloud} -d ./iaas/scs-0104-v1-images.yaml
            id: standard-images-check

Arguably, this is too arcane and not appropriate. We could introduce explicit parameters (that could then be listed in the docs, albeit in the docs for the certificate scope) like so:

      - name: Standard images
        url: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0104-v1-standard-images.md
        parameters:
          images_yaml: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Tests/iaas/scs-0104-v1-images.yaml
        checks:
          - executable: ./iaas/standard-images/images-openstack.py
            args: -c {os_cloud} -d {images_yaml}
            id: standard-images-check

In this case, we would need to debate the type of the parameter -- if we want it clickable in the docs, a URL like in this example would be good, but I would also want to avoid the additional roundtrip for the download in the test script (so maybe include a special case for URLs that point to the same repo).

Another point of debate would be the presentation in the docs -- currently we have this table:

Scope versions ->	v2	v3	v4
State	Deprecated	Effective	Effective
Stabilized at	2023-03-23	2023-06-15	2024-02-28
Deprecated at	2023-11-30	2024-04-30
Standards
OpenStack Powered Compute	v2022.11	v2022.11	v2022.11
scs-0100: Flavor naming	v2	v3	v3
scs-0101: Entropy			v1
scs-0102: Image metadata	v1	v1	v1
scs-0103: Standard flavors			v1
scs-0104: Standard images			v1

The parameters would go in here somewhere, but obviously we would need more space for them.

[josephineSei]

From reading the standard I first assumed, it will state the required images. But of course it can only state the procedure of how to get the CURRENTLY required images. This is totally right, because the standard should live way longer than the images, which could be outdated within a few months. So it would be good to have this pointed out more clearly in the standard: "We are standardizing a way to have the same images available at all scs-conformal clouds at the same time"

I agree with @markus-hentsch that we need to have a better way to present the yaml files (if they contain the images for the recent conformance tests / label). For example to pointing to the currently effective certificate scopes. This would also allow CSPs to check which yaml is needed for their scope and what to do to be able to "upgrade" to the next scope.

[markus-hentsch]

On second thought, the crux here is the following: from my point of view, this standard merely regulates the format of the YAML file; which YAML file is ultimately used is not part of the standard, but a parameter of the certificate subject. Well, apparently, this doesn't seem to be the most practical point of view... So this turns out to be something we should discuss in the SIG after all(!)

(bold emphasis added by me)

Currently, the source file classifies it as type "Standard"^1 whereas from my point of view, the majority of the content leans more towards the "Procedural" type as it explains a process rather than a standard that a CSP is to implement. But maybe I'm misinterpreting this classification. Seems like there is no easy/obvious way how to go forward with this.

I'll put it on the agenda for the next possible IaaS call[^2] for now.

[^2]: referring to 2024-06-05, since I'm unavailable on 2024-05-29

[mbuechse]

This issue should be mitigated by formal parameters. Nevertheless, we should also change this into a Procedural and add implementation notes.