search

kumahq / kuma

🐻 The multi-zone service mesh for containers, Kubernetes and VMs. Built with Envoy. CNCF Sandbox Project.
https://kuma.io/install
Apache License 2.0
3.67k stars 333 forks source link

Service reconciler injects annotations only when the Namespace is labeled. #11994

Open lukidzi opened 5 days ago

lukidzi commented 5 days ago

What happened?

Users can join the mesh by adding the kuma.io/sidecar-injection=true label to either the Namespace or the Deployment. When the label is applied at the Namespace level, a service reconciler adds the necessary annotations to the service to ensure it is configured for gateway usage. The annotations applied by the reconciler are:

    IngressServiceUpstream      = "ingress.kubernetes.io/service-upstream"
    NginxIngressServiceUpstream = "nginx.ingress.kubernetes.io/service-upstream"

When a gateway is deployed within a Namespace that has the kuma.io/sidecar-injection label, the reconciler adds these labels correctly. However, if the kuma.io/sidecar-injection label is added at the Deployment level rather than the Namespace level, the annotations are not applied as expected.

Expected behavior: Annotations should be injected when the kuma.io/sidecar-injection label is set at the Deployment level, ensuring consistent behavior regardless of where the label is applied.

lukidzi commented 5 days ago

There's no direct connection between a Deployment and a Service, so we can't reliably add annotations to the Service based solely on the presence of the kuma.io/sidecar-injection label on the Deployment. Additionally, the Deployment might be created after the Service, making it even harder to automatically sync annotations in such cases.

One solution could be to require users to manually add the necessary annotations to the Service when applying kuma.io/sidecar-injection at the Deployment level.

  1. If this approach is adopted, clear documentation is essential to guide users on manually adding the required labels and annotations to ensure consistent behavior.
  2. The user should additionally annotate Service with kuma.io/sidecar-injection and later we could add these labels automatically (check if that is not supported yet)
lukidzi commented 1 day ago

triage: we should document this better: Once a user has a kuma.io/sidecar-injection=true on the Deployment/ReplicaSet...., following annotations should be added manually:

  • "ingress.kubernetes.io/service-upstream"
  • "nginx.ingress.kubernetes.io/service-upstream"
jijiechen commented 1 hour ago

So there is no need to change Kuma code and the only thing we want to do is to update the doc website right?