loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.26k stars 398 forks source link

Kyverno pod mutate policy does not work with vcluster #211

Closed olljanat closed 2 years ago

olljanat commented 2 years ago

I trying to create Kyverno policy which overwrite resource requests for all pods running vcluster namespaces (because many of them request much more than they really need).

Issue is that this policy works for pods created from host cluster but not for pods created by syncer:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: set-resource-requests
spec:
  validationFailureAction: enforce
  background: false
  rules:
  - name: set-resource-requests
    match:
      resources:
        kinds:
        - Pod
        namespaces:
        - "vcluster-*"
    preconditions:
      all:
      - key: "{{request.operation}}"
        operator: In
        value:
        - CREATE
        - UPDATE
    mutate:
      foreach:
      - list: "request.object.spec.containers"
        patchStrategicMerge:
          spec:
            containers:
            - name: "{{ element.name }}"
              resources:
                requests:
                  cpu: "10m"
                  memory: 10Mi

What I don't understand is that why this is the case? Afaiu syncer call host cluster kube-apiserver so Kyverno running on host cluster should see those events.

FabianKramm commented 2 years ago

@olljanat thanks for creating this issue! Yes I don't see a reason why this shouldn't work as vcluster just calls the host server's api server to create any pods.

olljanat commented 2 years ago

OK. Then this is most probably issue on Kyverno side. Thanks