kubernetes-sigs / kustomize

Customization of kubernetes YAML configurations
Apache License 2.0
10.7k stars 2.22k forks source link

Usage of `namePrefix` or `nameSuffix` with Validating Admission Policy results in a silently broken `ValidatingAdmissionPolicyBinding` #5674

Closed m1kola closed 3 weeks ago

m1kola commented 2 months ago

What happened?

Validating Admission Policy APIs do not seem to be supported at the moment by namePrefix and nameSuffix fields.

As a result manifests that contain ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding pair are being partially transformed: their names are being changed however ValidatingAdmissionPolicyBinding ends up referencing ValidatingAdmissionPolicy without the prefix/suffix in .spec.policyName field.

Resulting ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding can still be applied to the cluster, but they will have no effect on admission.

What did you expect to happen?

Name reference in .spec.policyName gets updated.

How can we reproduce it (as minimally and precisely as possible)?

Consider the following kustomization.yaml:

namePrefix: "silently-wont-bind-"

resources:
- admission.yaml

And admission.yaml:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicy
metadata:
  name: "example-policy"
spec:
  failurePolicy: Fail
  paramKind:
    apiVersion: apps/v1
    kind: Deployment
  matchConstraints:
    resourceRules:
    - apiGroups:   ["apps"]
      apiVersions: ["v1"]
      operations:  ["CREATE", "UPDATE"]
      resources:   ["deployments"]
  validations:
    - expression: "!object.metadata.name.startsWith('my-')"
      message: "Example admission policy! Deployment name must not start with my-"
      reason: Invalid

---

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: "example-policy-binding"
spec:
  policyName: "example-policy"
  validationActions: [Deny]

Expected output

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicy
metadata:
  name: silently-wont-bind-example-policy
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups:
      - apps
      apiVersions:
      - v1
      operations:
      - CREATE
      - UPDATE
      resources:
      - deployments
  paramKind:
    apiVersion: apps/v1
    kind: Deployment
  validations:
  - expression: '!object.metadata.name.startsWith(''my-'')'
    message: Example admission policy! Deployment name must not start with my-
    reason: Invalid
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: silently-wont-bind-example-policy-binding
spec:
  policyName: silently-wont-bind-example-policy
  validationActions:
  - Deny

Actual output

After kustomize build this results in the following:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicy
metadata:
  name: silently-wont-bind-example-policy
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
    - apiGroups:
      - apps
      apiVersions:
      - v1
      operations:
      - CREATE
      - UPDATE
      resources:
      - deployments
  paramKind:
    apiVersion: apps/v1
    kind: Deployment
  validations:
  - expression: '!object.metadata.name.startsWith(''my-'')'
    message: Example admission policy! Deployment name must not start with my-
    reason: Invalid
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: silently-wont-bind-example-policy-binding
spec:
  policyName: example-policy
  validationActions:
  - Deny

Note: this output is still can be applied to the cluster without any errors. But the admission policy will not have any effect. This can be tested with this deployment (policy denies deployments prefixed with my-):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Kustomize version

v5.4.1

Operating system

MacOS

m1kola commented 2 months ago

Forgot to mention. I think there is a workaround.

Replace:

namePrefix: "silently-wont-bind-"

With:

transformers:
- |-
  apiVersion: builtin
  kind: PrefixSuffixTransformer
  metadata:
    name: prefix-all-names
  prefix: silently-wont-bind-
  fieldSpecs:
    - path: metadata/name
    - group: admissionregistration.k8s.io
      kind: ValidatingAdmissionPolicyBinding
      path: spec/policyName
stormqueen1990 commented 2 months ago

/assign

m1kola commented 2 months ago

I think this ideally should be addressed in Kustomize itself because Validating Admission Policy APIs are now stable starting Kubernetes 1.30. But for these who are looking to make it work right now here is an alternative workaround with nameReference:

Assuming admission.yaml is the same as reported in the issue, update kustomization.yaml to be:

configurations:
- kustomizeconfig.yaml

namePrefix: "silently-wont-bind-"

resources:
- admission.yaml

And introduce kustomizeconfig.yaml:

nameReference:
- kind: ValidatingAdmissionPolicy
  group: admissionregistration.k8s.io
  fieldSpecs:
  - kind: ValidatingAdmissionPolicyBinding
    group: admissionregistration.k8s.io
    path: spec/policyName
stormqueen1990 commented 2 months ago

Hi there, @m1kola! This makes sense to me. I'm planning to take a look at what it would take to update Kustomize for this later today.

/triage accepted