open-policy-agent / opa

Open Policy Agent (OPA) is an open source, general-purpose policy engine.
https://www.openpolicyagent.org
Apache License 2.0
9.62k stars 1.33k forks source link

OPA policy is valid but failing in kubernetes #4789

Closed techiescorner closed 2 years ago

techiescorner commented 2 years ago

Expected Behavior The admission controller returns an error and blocks resource creation.

Actual Behavior Admission controller passes policy evaluation and creates incompliant resources

Steps to Reproduce the Problem Rego Policy:

https://play.openpolicyagent.org/p/EEOfeSQHFo Playground doesn't show any error to the policy but it is not validating in the Kubernetes

Apply the template and constraint file and then try to create an Ingress with the "nginx.ingress.kubernetes.io/limit-connections" annotations with values other than 5 or 10. It should fail with an error.

Kubernetes Manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: opa-test
  annotations:
    nginx.ingress.kubernetes.io/limit-connections: "8"
spec:
  rules:
    - host: hello-world.info
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 8080

OPA template file

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
  namespace: opa
spec:
  crd:
    spec:
      names:
        kind: k8srequiredlabels
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package kubernetes.admission
        operations = {"CREATE", "UPDATE"}
        required_annotations = {"nginx.ingress.kubernetes.io/limit-connections": {"5", "10"}, "nginx.ingress.kubernetes.io/limit-rps": {"5"}}

        violation[{"msg": msg}] {
          input.request.kind.kind = "Ingress"
          operations[input.request.operation]
          required_annotations[key]
          not input.request.object.metadata.annotations[key]
          msg := sprintf("Compliance check failed:  wrong Rate_limit value provided. Missing annotation %v required", [key])
         }

        violation[{"msg": msg}] {
          input.request.kind.kind = "Ingress"
          operations[input.request.operation]
          possible_values := required_annotations[key]
          value := input.request.object.metadata.annotations[key]
          not possible_values[value]
          msg := sprintf("Compliance check failed: rate_limiting annotation is missing. Bad annotation key-value: %v = %v", [key, value])
        }

OPA constraint file

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: k8srequiredlabels
metadata:
  name: nginx-rate-limit
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: [""]
    namespaces:
      - "opa-test"

Ensure OPA template and constraint is installed

k apply -f template.yaml constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels configured k apply -f constraint.yaml k8srequiredlabels.constraints.gatekeeper.sh/nginx-rate-limit configured

Gatekeeper is running $ k get pods -n gatekeeper-system NAME READY STATUS RESTARTS
gatekeeper-audit-6d467bddf-h2ftq 1/1 Running 0
gatekeeper-controller-manager-8fd44f7d8-5cp65 1/1 Running 0
gatekeeper-controller-manager-8fd44f7d8-qzzdx 1/1 Running 0
gatekeeper-controller-manager-8fd44f7d8-tc8m2 1/1 Running 0

Additional Info Using image: openpolicyagent/opa:0.10.5 Using image: openpolicyagent/kube-mgmt:0.6

jacmillan commented 1 year ago

Hi, Was this issue resolved, could you please guide what was missing. I am trying to create an Azure policy to restrict Ingress controller to Nginx.