Azure / application-gateway-kubernetes-ingress

This is an ingress controller that can be run on Azure Kubernetes Service (AKS) to allow an Azure Application Gateway to act as the ingress for an AKS cluster.
https://azure.github.io/application-gateway-kubernetes-ingress
MIT License
667 stars 413 forks source link

Using Azure Waf Policy For Path on a specific ingress when policy doesn't exist breaks entire app gateway #1615

Open Poltergeisen opened 1 month ago

Poltergeisen commented 1 month ago

Describe the bug We have an app gateway that is shared for an entire AKS cluster. When a team accidentally references a WAF policy that doesn't exist yet, it breaks the app gateway backend connections for the entire App Gateway, and not for the ingress in question.

To Reproduce Steps to reproduce the behavior: Create an AKS cluster Create 2 applications to run on the cluster, one using no WAF policy and one using a WAF policy that doesn't exist yet

Both applications fail to resolve IP addresses from the backend pool.

Example ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-api
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    cert-manager.io/cluster-issuer: sectigo-issuer
    appgw.ingress.kubernetes.io/backend-protocol: http
    appgw.ingress.kubernetes.io/use-private-ip: "true"
    appgw.ingress.kubernetes.io/waf-policy-for-path: "subscriptions/xxx-xxx-xxx/resourceGroups/my-rg/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/my-policy-that-doesnt-exist"
spec:
  tls:
  - hosts:
    - my.host.tld
    secretName: sbx-tls
  rules:
  - host:my.host.tld
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port: 
              number: 80

Ingress Controller details

erkkov2lja commented 1 week ago

The issue is there and will break the whole AKS ingresses created, after one of the ingress rules would have non-existing WAF rule reference.

waf_issue

Poltergeisen commented 1 week ago

The issue is there and will break the whole AKS ingresses created, after one of the ingress rules would have non-existing WAF rule reference.

waf_issue

Yes, this is what we experienced. We are working with a CSA on this, but there doesn't seem to be any good solutions. Waiting to hear back more and can update this issue if I do