[Module catalogue improvements ] Introduce New ModuleTemplate API version to support the new Module Catalog

Description

Note: This issue is just about API change, the change in the controller logic is expected to be implemented here

As described here, ModuleTemplate CR must be extended.

What we essentially want follows the ideas from the community-modules repository. In general, we want that:

Every ModuleTemplate describes a single module version
Every such version may be selected by user(s) for installation
Every such version may be assigned to a set of channels - possibly empty. Module Team decides about versions-channels mapping.

In addition, to support Deletion modes we need a managedResources list describing the list of Group/Version/Kinds that are defined by the Module's Team as "managed" by the module. There is a decision record for that. Assuming that list is embedded, no further API changes are required on the ModuleTemplate CR.

The set of ModuleTemplate CRs in the control-plane defines the Module Catalog for the Lifecycle-Manager.

The above requirements can be implemented by releasing a new ModuleTemplate API version with the following changes:

Deprecate the channel attribute
Add an optional version attribute. At the moment it is an annotation: "operator.kyma-project.io/module-version": "0.1.2". This attribute will be required in the future
Add an optional name attribute. At the moment it is a label: "operator.kyma-project.io/module-name": "template-operator". It is not strictly required, it's just to make the API more robust. It will be required in the future.

Reasons

We need that to support features described in #1472

The Data Model

Example: Consider a module: foo. It will be released in two versions available for user to install: 1.0.0, 2.0.0. Additionally there is a 2.2.0 version that is only available in the experimental channel. The 1.0.0 is released in the regular channel, the 2.0.0 version is released in the fast channel

We need five ModuleTemplate CRs to model that:

ModuleTemplate for the version 1.0.0

name: "foo"
version: "1.0.0"
descriptor: [ OCM descriptor for version 1.0.0 ]

ModuleTemplate for the version 2.0.0

name: "foo"
version: "2.0.0"
descriptor: [ OCM descriptor for version 2.0.0 ]

ModuleTemplate for the regular channel

name: "foo"
version: "regular"
descriptor: [ OCM descriptor for version 1.0.0 ]

ModuleTemplate for the fast channel

name: "foo"
version: "fast"
descriptor: [ OCM descriptor for version 2.0.0 ]

ModuleTemplate for the experimental channel

name: "foo"
version: "experimental"
descriptor: [ OCM descriptor for version 2.2.0 ]

After some time, the team decides to upgrade the module in the regular channel and also make the version 2.2.0 explicitly available for users. In order to do that the module team must:

Update the ModuleTemplate for the regular channel:

name: "foo"
version: "regular"
descriptor: [ OCM descriptor for version 2.0.0 ]

Create a new ModuleTemplate for the 2.2.0 version:

name: "foo"
version: "2.2.0"
descriptor: [ OCM descriptor for version 2.2.0 ]

In this model, the channel is just another "version" to choose from. The channel doesn't refer to any other ModuleTemplate CR, it only references an OCM artifact - like any other version. It is responsibility of the Team that manages the module to take care that the ModuleTemplate with a channel as a version refers to a proper OCM artifact. In other words the mapping of "channel" to "module version" is implicit here. Yet, the desired feature of having a channel that is a "label" on a single module version is achieved. For example, a ModuleTemplate for some module that uses regular as a version may refer to the same OCM artifact as a ModuleTemplate for the same module that uses 1.0.0 as a version. It may also refer to the OCM artifact for which there is no explicit "versioned" ModuleTemplate, yet it is still some version - because it exists as an OCM artifact so it was released.

Single attribute vs two attributes

The same model can be expressed in two ways:

having a single version attribute that can take values like 1.0.0, 2.0.0, regular, fast etc.
having a separate channel attribute (regular, fast) and a separate version attribute (1.0.0, 2.0.0) with additional validation that only one of these may be defined - the other one must be empty.

A single attribute solution is simpler, but if this cannot be easily validated (a string that is either semantic version OR arbitrary token), the double-attribute solution is an option.

Acceptance Criteria

[ ] Introduce new API version
[ ] Extend the model with a version attribute. the value must be a semantic version OR a channel reference: regular, fast, experimental etc. Alternatively, see the Single attribute vs two attributes note above.
[ ] Add an explicit name attribute (currently it is a label).

Feature Testing

e2e tests for validation on the new API version.

Testing approach

No response

Attachments

No response

Related issue(s)

https://github.com/kyma-project/lifecycle-manager/issues/1472

We may also consider to introduce a new CR: ModuleVersion, that would take over the existing ModuleTemplate Reasons:

The name better corresponds to the role the CR serves (it describes an entry in the Module Catalog for a given module version)
Less confusion about how the "old" vs "new" ModuleTemplate works

One thing we need to consider is how the installed module version shows in Busola. Right now, the module overview of a given Kyma shows the active channel and currently installed version of the module. If we remove the channel and use regular or fast version identifiers in the same way as we use the semantic versions, the user should still be able to see the exact version that is currently installed.

See also https://github.com/kyma-project/lifecycle-manager/issues/1472#issuecomment-2165801235

EDIT: clarified with Xin. Channel concept overall stays in place. It is just our internal representation of ModuleTemplate that changes.

Xin and myself discussed this a bit further...

AC: Extend the model with a version attribute. the value must be a semantic version OR a channel reference: regular, fast, experimental etc. Alternatively, see the Single attribute vs two attributes note above.

If choosing a single-attribute approach, we should not name the field version. Values like regular or fast are not versions and have a special meaning. Rather name it tag or something different, open for proposals.

Note that in KymaCR, we would still go for separated version and channel to keep the channel concept as known by the end user. Just internally we translate both to tag in the ModuleTemlate (https://github.com/kyma-project/lifecycle-manager/issues/1589).

AC: Add an explicit name attribute (currently it is a label).

We should also state explicitly that the resource name must follow the convention: <module-name>-<tag>. ~~tag is alpha numeric and MUST NOT include dashes (in case we need to split it again in the future).~~ This will help us simplifying the currently rather complex template lookup as we can do something like:

tmplt := &v1beta2.ModuleTemplate{}
err := c.Get(context.Background(), client.ObjectKey{
        Namespace: namespace,
        Name:      fmt.Sprintf("%s-&s", moduleName, moduleTag),
    }, tmplt)

Even though we have the resource name convention, module-name and tag should still be explicit attributes on the spec.

An example for a full ModuleTemplate with one attribute approach:

apiVersion: operator.kyma-project.io/v1beta3
kind: ModuleTemplate
metadata:
  name: template-operator-1.1.0
  namespace: kcp-system
spec:
  module-name: template-operator
  tag: 1.1.0
  data:
    // ...
  descriptor:
    // ... OCM descriptor for version 1.1.0
  mandatory: false
---
// given the current version for fast is also 1.1.0
apiVersion: operator.kyma-project.io/v1beta3
kind: ModuleTemplate
metadata:
  name: template-operator-fast
  namespace: kcp-system
spec:
  module-name: template-operator
  tag: fast
  data:
    // ...
  descriptor:
    // ... OCM descriptor for version 1.1.0
  mandatory: false

An example for a full ModuleTemplate with two attribute approach:

apiVersion: operator.kyma-project.io/v1beta3
kind: ModuleTemplate
metadata:
  name: template-operator-1.1.0
  namespace: kcp-system
spec:
  module-name: template-operator
  version: 1.1.0
  data:
    // ...
  descriptor:
    // ... OCM descriptor for version 1.1.0
  mandatory: false
---
// given the current version for fast is also 1.1.0
apiVersion: operator.kyma-project.io/v1beta3
kind: ModuleTemplate
metadata:
  name: template-operator-fast
  namespace: kcp-system
spec:
  module-name: template-operator
  channel: fast
  data:
    // ...
  descriptor:
    // ... OCM descriptor for version 1.1.0
  mandatory: false

We should also state explicitly that the resource name must follow the convention: -. tag is alpha numeric and MUST NOT include dashes (in case we need to split it again in the future).

If tag is a version, then we can't prevent it from including dashes, patch version appending hyphen symbol is so-called pre-release version, Examples: 1.0.0-alpha, 1.0.0-alpha. check spec 9

Even though we have the resource name convention, name and tag should still be explicit attributes on the spec.

the name better renamed to module-name

If tag is a version, then we can't prevent it from including dashes, patch version appending hyphen symbol is so-called pre-release version, Examples: 1.0.0-alpha, 1.0.0-alpha. check spec 9

True. Dash (-) and dot (.) are the only allowed special chars for resource names. We could change to dot to be on the safe side if we ever need to parse name and version/channel out of a concatenated resource name again. However, I doubt that we will need this since we also explicitly add both to the spec. So for my take, we could drop the requirement to not include dashes.

the name better renamed to module-name

👍🏻

kyma-project / lifecycle-manager