loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.16k stars 372 forks source link

Invalid DownwardAPI metadata.labels in StatefulSet Pods as of v0.3.0-beta.3 #50

Closed nktpro closed 3 years ago

nktpro commented 3 years ago

As of the latest release of v0.3.0-beta.3, DownwardAPI metadata.labels for StatefulSet Pods would reference wrong label names. Here's a minimum reproducer:

apiVersion: v1
kind: Namespace
metadata:
  name: vcluster-bug
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: vcluster-bug
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
  namespace: vcluster-bug
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: k8s.gcr.io/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: web
        env:
          - name: K8S_STATEFUL_SET_POD_NAME
            valueFrom:
              fieldRef:
                apiVersion: v1
                fieldPath: metadata.labels['statefulset.kubernetes.io/pod-name']

A Pod launched inside the host cluster would then look like this:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    vcluster.loft.sh/hosts-rewritten: "true"
    vcluster.loft.sh/labels: |-
      app="nginx"
      controller-revision-hash="web-5d76df5b6d"
      statefulset.kubernetes.io/pod-name="web-0"
    vcluster.loft.sh/name: web-0
    vcluster.loft.sh/namespace: vcluster-bug
    vcluster.loft.sh/owner-set-kind: StatefulSet
    vcluster.loft.sh/service-account-name: default
    vcluster.loft.sh/uid: c683994e-91ab-4343-b0cf-d22bd6916a51
  creationTimestamp: "2021-06-18T08:28:36Z"
  labels:
    vcluster.loft.sh/label-my-vcluster-x-67b9ac9205: web-5d76df5b6d
    vcluster.loft.sh/label-my-vcluster-x-a172cedcae: nginx
    vcluster.loft.sh/label-my-vcluster-x-ec0342c544: web-0
    vcluster.loft.sh/managed-by: my-vcluster
    vcluster.loft.sh/namespace: vcluster-bug-x-my-vcluster
  name: web-0-x-vcluster-bug-x-my-vcluster
  namespace: my-vcluster
  ownerReferences:
  - apiVersion: apps/v1
    kind: StatefulSet
    name: my-vcluster
    uid: 0a0910eb-e109-4637-9557-93cfec84cd1b
  resourceVersion: "48331478"
  uid: e94a26be-c551-4a05-9cb3-56b0615bf9db
spec:
  automountServiceAccountToken: false
  containers:
  - env:
    - name: K8S_STATEFUL_SET_POD_NAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.labels['vcluster.loft.sh/label-my-vcluster-x-4992f265ef']
... [REDACTED] ...

Notice that the rewritten fieldPath is incorrect. It's referencing a non-existent label of vcluster.loft.sh/label-my-vcluster-x-4992f265ef, instead of vcluster.loft.sh/label-my-vcluster-x-ec0342c544, which corresponds to the original statefulset.kubernetes.io/pod-name label.

FabianKramm commented 3 years ago

@nktpro thanks for creating this issue! Yes you are correct, just verified this, we'll fix that

FabianKramm commented 3 years ago

@nktpro should work now with v0.3.0-beta.4