search

kyverno / kyverno

Cloud Native Policy Management
https://kyverno.io
Apache License 2.0
5.8k stars 893 forks source link

cel policy blocks deletion of resources [Bug] #10745

Open anushkamittal2001 opened 4 months ago

anushkamittal2001 commented 4 months ago

Kyverno Version

1.12.5

Kubernetes Version

1.27.x

Kubernetes Platform

KinD

Kyverno Rule Type

Validate

Description

After applying cel policy in enforce mode, a good resource gets accepted, a bad resource gets blocked. But when I try to delete the good resource. it get blocked.

Steps to reproduce

  1. policy 
    apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: restrict-binding-system-groups
annotations:
policies.kyverno.io/title: Restrict Binding System Groups in CEL expressions
policies.kyverno.io/category: RBAC Best Practices in CEL 
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: RoleBinding, ClusterRoleBinding, RBAC
policies.kyverno.io/minversion: 1.11.0
kyverno.io/kubernetes-version: "1.26"
policies.kyverno.io/description: >-
  Certain system groups exist in Kubernetes which grant permissions that
  are used for certain system-level functions yet typically never appropriate
  for other users. This policy prevents creating bindings for system:masters group.      
spec:
validationFailureAction: Enforce
background: true
rules:
- name: restrict-masters
  match:
    any:
    - resources:
        kinds:
          - RoleBinding
          - ClusterRoleBinding
  validate:
    cel:
      expressions:
        - expression: "object.roleRef.name != 'system:masters'"
          message: "Binding to system:masters is not allowed."
  2. good res 
    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: goodcrb01
subjects:
- kind: Group
name: secret-reader
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: manager
apiGroup: rbac.authorization.k8s.io
  3. bad resource 
    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: badcrb01
subjects:
- kind: Group
name: bar
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: "system:masters"
apiGroup: rbac.authorization.k8s.io

Result 

Downloads ) k apply -f /Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac.yaml                                                                                                  
clusterpolicy.kyverno.io/restrict-binding-system-groups created
Downloads ) k apply -f /Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac-good-res.yaml                                                                                         
clusterrolebinding.rbac.authorization.k8s.io/goodcrb01 created
Downloads ) k apply -f /Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac-bad-res.yaml                                                                                          
Error from server: error when creating "/Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac-bad-res.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

resource ClusterRoleBinding//badcrb01 was blocked due to the following policies 

restrict-binding-system-groups:
  restrict-masters: Binding to system:masters is not allowed.
Downloads ) k delete -f /Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac-good-res.yaml                                                                                        
Error from server: error when deleting "/Users/nirmata/Desktop/go-nctl/anushka-test/cel-pol-rbac-good-res.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: 

resource ClusterRoleBinding//goodcrb01 was blocked due to the following policies 

restrict-binding-system-groups:
  restrict-masters: 'expression ''object.roleRef.name != ''system:masters'''' resulted
    in error: no such key: roleRef'

Expected behavior

I should have seen the good resource get deleted.

Screenshots

No response

Kyverno logs

No response

Slack discussion

No response

Troubleshooting

MariamFahmy98 commented 4 months ago

You should set the operations to CREATE and UPDATE as follows:

- resources:
    kinds:
      - RoleBinding
      - ClusterRoleBinding
    operations:
      - CREATE
      - UPDATE

This is because there are recent changes to apply cel policies to a deleted resource so by default, all operations are set.

Let me know if this solves the issue.

anushkamittal2001 commented 4 months ago

Let me check this out. Thanks @MariamFahmy98

anushkamittal2001 commented 4 months ago

@MariamFahmy98 Could you help me understand why the good pod gets created but doesnt get deleted.

If it is a good pod both should be allowed right

MariamFahmy98 commented 4 months ago

@MariamFahmy98 Could you help me understand why the good pod gets created but doesnt get deleted.

If it is a good pod both should be allowed right

By not setting operations, the policy defaults to all operations. Therefore, when deleting the pod, the policy is applied and the CEL expression fails with the error no such key: roleRef'. This is because the resource is deleted and there is no roleRef field so it is important to set operations to CREATE and UPDATE to avoid applying policies on deleted resource.

anushkamittal2001 commented 3 months ago

Thanks @MariamFahmy98! Is there documentation for this that you can point me to?

dosubot[bot] commented 3 weeks ago

Hi, @anushkamittal2001. I'm helping the kyverno team manage their backlog and am marking this issue as stale.

Your issue highlights a bug in Kyverno version 1.12.5, where a CEL policy in enforce mode is preventing the deletion of a valid ClusterRoleBinding resource. You mentioned seeking further clarification and documentation on this behavior, and MariamFahmy98 suggested adjusting the policy to specify only CREATE and UPDATE operations to avoid applying the policy during deletion, which resulted in an error due to the absence of the roleRef field.

Could you please let us know if this issue is still relevant to the latest version of the kyverno repository? If it is, feel free to comment here to keep the discussion going. Otherwise, you can close the issue yourself, or it will be automatically closed in 7 days. Thank you!

Moglum commented 2 weeks ago

This issue is still present in v 1.13.0 (Helm Chart version 3.3.2).

What makes this more confusing is that the documentation explicitly mentions

The operations[] list is optional but recommended. When operations[] is absent, the default behavior is to match on CREATE and UPDATE requests.

Which is true for policies NOT using CEL.

The CEL-specific documentation doesn't mention any difference in behavior

dosubot[bot] commented 2 weeks ago

@realshuting, the user has indicated that this issue is still present in v 1.13.0 and has provided additional context regarding the confusion with the documentation on the operations list and CEL behavior. Could you please assist them with this issue?