Consistent labeling of kyma modules

[pbochynski]

Created on 2023-12-05} by Piotr Bochynski(@pbochynski)

Decision log

Name	Description
Title	Consistent labeling
Due date	2023-12-19
Status	Proposed on 2023-12-05, Accepted on 2024-01-02
Decision type	Binary
Affected decisions	-

Context

It should be possible to identify kubernetes resources that belong to the kyma module. Tools like Kyma Dashboard can use labels to show module healthiness or resource consumption.

Decision

All module resources should have a label:

kyma-project.io/module={module_name}

If module is deployed in own namespace it should also have such label. The kyma-system namespace is shared across mutliple modules, therefore I propose to label it with kyma-project.io/module=kyma for that namespace, but modules deployed inside should have proper label like kyma-project.io/module=eventing or kyma-project.io/module=serverless

Apart from that label all components should use properly common labels. Example for serverless-manager deployment:

    app.kubernetes.io/name: serverless-manager
    app.kubernetes.io/instance: serverless-manager
    app.kubernetes.io/version: "1.0.3"
    app.kubernetes.io/component: controller
    app.kubernetes.io/part-of: serverless

[k15r]

@pbochynski i do not understand the purpose of kyma-project.io/module=kyma for modules in the kyma-system namespace. Could you please elaborate?

imho even modules in the kyma-system namespace should have eg. kyma-project.io/module=eventing to specify the actual module the resource is a part of?

another question is related to the part-of label. currently this seems to be set by the klm and its value is app.kubernetes.io/part-of=Kyma. Will this in future not override the values anymore?

[k15r]

Could you please comment on this set of labels:

kyma-project.io/module: eventing
app.kubernetes.io/managed-by: kyma-lifecycle-manager # for resources created by klm using the module-manifest, otherwise app.kubernetes.io/created-by: eventing-manager
app.kubernetes.io/instance: eventing-manager # for resources created by the module itself it should be a unique name based on the CR owning the created resources
app.kubernetes.io/name: eventing # how is this different from `kyma-project.io/module=eventing`
app.kubernetes.io/part-of: Kyma
app.kubernetes.io/version: "1.0.3"
app.kubernetes.io/component: controller
operator.kyma-project.io/managed-by: declarative-v2
operator.kyma-project.io/watched-by: module-manager

[pbochynski]

About kyma-system I updated the description. It is only about namespace itself (not the content).

About app.kubernetes.io/name and kyma-project.io/module Not all modules contain only one application. The common label app.kubernetes.io/name is not the same for all the components that contribute to telemetry module for example. In simple cases kyma-project.io/module will have the same value as app.kubernetes.io/name.

About app.kubernetes.io/part-of I do not have strong opinion. It can be kyma, it can be module name. What is your preference? We just should align.

In general I am open to suggestions about values. The consistency is important here.

[k15r]

Thanks for clarification about the namespace. Regarding part-of: I'm fine with the current implementation in klm but could you please update the description to provide a consistent example. The current value of part-of will always be changed by klm whenever a resource is deployed. So any value previously specified for part-of cannot be found in the cluster.

[a-thaler]

I like the proposal. To have it more illustrative, could you please extend your example with the simple and advanced scenario. Indeed serverless is a good complex example as it consist of 4 parts:

• serverless-ctrl-mngr
• serverless-docker-registry
• serverless-operator
• serverless-webhook

[pbochynski]

Example for serverless.

serverless-operator:
  kyma-project.io/module: serverless
  app.kubernetes.io/name: serverless-operator
  app.kubernetes.io/instance: serverless-operator-default
  app.kubernetes.io/version: 1.2.1
  app.kubernetes.io/component: operator
  app.kubernetes.io/part-of: serverless
  app.kubernetes.io/managed-by: 

serverless-ctrl-mngr:
  kyma-project.io/module: serverless
  app.kubernetes.io/name: serverless-controller
  app.kubernetes.io/instance: serverless-controller-default
  app.kubernetes.io/version: 1.2.1
  app.kubernetes.io/component: controller,
  app.kubernetes.io/part-of: serverless
  app.kubernetes.io/managed-by: 

serverless-docker-registry:
  kyma-project.io/module: serverless
  app.kubernetes.io/name: serverless-docker-registry
  app.kubernetes.io/instance: serverless-docker-registry-default
  app.kubernetes.io/version: 2.8.1
  app.kubernetes.io/component: docker-registry
  app.kubernetes.io/part-of: serverless
  app.kubernetes.io/managed-by: 

serverless-webhook-svc:
  kyma-project.io/module: serverless
  app.kubernetes.io/name: serverless-webhook
  app.kubernetes.io/instance: serverless-controller-default
  app.kubernetes.io/version: 1.2.1
  app.kubernetes.io/component: webhook
  app.kubernetes.io/part-of: serverless
  app.kubernetes.io/managed-by: 

Managed-by should be empty in the deployment yaml. In the runtime it should be set by the Kyma Lifecycle Manager.

[pbochynski]

No objections - proposal accepted.

[abbi-gaurav]

@pbochynski One of the requirements from the customer to be able to identify all the resources that are created / owned / managed by Kyma. That way, they can exclude such resources (e.g. ClusterRoles, ClusterRoleBindings and others) from their policies.

Something along the lines

app.kubernetes.io/managed-by: btp (or kyma or kyma-life-cycle-manager)

Main is it should be uniform and single so it makes it easier on the customer side to whitelist such resources with a single rule.

[a-thaler]

@pbochynski can we please include the feedback of @abbi-gaurav please. We could add something like app.kubernetes.io/managed-by: kyma-lifecycle-manager, however it is not really true for resources managed by a module manager. Is it maybe good enough for customers to check for presence of the kyma-project.io/module label

[Ressetkk]

I've got familiar with the official recommendations from K8s and I believe that the better place to handle setting app.kubernetes.io/instance label would be the same as with app.kubernetes.io/managed-by one - through KLM. This label describes single instance of an application, like unique deployment. This shouldn't be hardcoded in the YAML, and rather I'd take example how ArgoCD handles that - https://argo-cd.readthedocs.io/en/stable/faq/#why-is-my-app-out-of-sync-even-after-syncing

Argo CD automatically sets the app.kubernetes.io/instance label and uses it to determine which resources form the app.

Quick idea would be to define small patch before module apply and set this to name-{{ .uniqueSuffix }} during deployment on all resources in manifest....

maybe handling this label could be thought by @kyma-project/jellyfish?