SSL Redirect not applied when appgw.ingress.kubernetes.io/ssl-redirect: 'true'

thisispaulsmith commented 2 years ago

Describe the bug When adding ingress with appgw.ingress.kubernetes.io/ssl-redirect: 'true' a redirect is not created. Instead ingress rules for https and http are created with no redirect. I've seen other issues that are similar but nothing with a resolution. We are using the latest version of AGWIC. The defined secret does exist. The logs refer to a redirectConfigurations that doesn't exist?

To Reproduce Deploy the ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    appgw.ingress.kubernetes.io/ssl-redirect: 'true'
    cert-manager.io/cluster-issuer: letsencrypt
    kubernetes.io/ingress.class: azure/application-gateway
spec:
  tls:
    - hosts:
        - testing.domain.com
      secretName: testing-tls
  rules:
  - http:
      paths:
        - path: /testpath
          pathType: ImplementationSpecific
          backend:
            service:
              name: test
              port:
                name: http
        - path: /testpath/*
          pathType: ImplementationSpecific
          backend:
            service:
              name: test
              port:
                name: http

Ingress Controller details

Name:         ingress-azure-78d65b5cd-nmmpz
Namespace:    ingress-azure
Priority:     0
Node:         aks-primarypool-15941712-vmss000000/15.0.0.115
Start Time:   Tue, 17 May 2022 16:24:03 +0100
Labels:       aadpodidbinding=ingress-azure
              app=ingress-azure
              pod-template-hash=78d65b5cd
              release=ingress-azure
Annotations:  checksum/config: 3e762fc7679a8bfe202d7ee7c7e5c1a64cd1c7628a8f13409542bbdb1bffc796
              prometheus.io/port: 8123
              prometheus.io/scrape: true
Status:       Running
IP:           15.0.0.161
IPs:
  IP:           15.0.0.161
Controlled By:  ReplicaSet/ingress-azure-78d65b5cd
Containers:
  ingress-azure:
    Container ID:   containerd://fd1ce480577b4f5930c4ca447af9d8384450269b2f7f9591c0cbdf557736b001
    Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.1
    Image ID:       mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:cc131292df265926942e23ca5601a3de66e8feabcb81f705d8f7d84b740f81b6
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 17 May 2022 16:24:04 +0100
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:      http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      ingress-azure  ConfigMap  Optional: false
    Environment:
      AZURE_CLOUD_PROVIDER_LOCATION:  /etc/appgw/azure.json
      AGIC_POD_NAME:                  ingress-azure-78d65b5cd-nmmpz (v1:metadata.name)
      AGIC_POD_NAMESPACE:             ingress-azure (v1:metadata.namespace)
    Mounts:
      /etc/appgw/ from azure (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lckjc (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  azure:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/kubernetes/
    HostPathType:  Directory
  kube-api-access-lckjc:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  28m   default-scheduler  Successfully assigned ingress-azure/ingress-azure-78d65b5cd-nmmpz to aks-primarypool-15941712-vmss000000
  Normal  Pulling    28m   kubelet            Pulling image "mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.1"
  Normal  Pulled     28m   kubelet            Successfully pulled image "mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.1" in 113.869531ms
  Normal  Created    28m   kubelet            Created container ingress-azure
  Normal  Started    28m   kubelet            Started container ingress-azure

Logs specfic to the problem

E0517 15:50:06.038231       1 requestroutingrules.go:264] Will not attach default redirect to rule; SSL Redirect does not exist: /subscriptions/a3c8c6b8-cc0e-48a4-94b7-daf0bc899a8b/resourceGroups/***/providers/Microsoft.Network/applicationGateways/***/redirectConfigurations/sslr-fl-a4f124d94a7f828fc2d0d455d1dc6d32
E0517 15:50:06.038361       1 requestroutingrules.go:355] Will not attach redirect to rule; SSL Redirect does not exist: /subscriptions/a3c8c6b8-cc0e-48a4-94b7-daf0bc899a8b/resourceGroups/***/providers/Microsoft.Network/applicationGateways/***/redirectConfigurations/sslr-fl-a4f124d94a7f828fc2d0d455d1dc6d32
E0517 15:50:06.038466       1 requestroutingrules.go:355] Will not attach redirect to rule; SSL Redirect does not exist: /subscriptions/a3c8c6b8-cc0e-48a4-94b7-daf0bc899a8b/resourceGroups/***/providers/Microsoft.Network/applicationGateways/***/redirectConfigurations/sslr-fl-a4f124d94a7f828fc2d0d455d1dc6d32

Any Azure support tickets associated with this issue. There is no associated ticked.

akshaysngupta commented 2 years ago

@thisispaulsmith can you also add host in the rules as well. Example:

...
 tls:
    - hosts:
        - testing.domain.com
      secretName: testing-tls
  rules:
  - host: testing.domain.com
    http:
    paths:
        - path: /testpath
          pathType: ImplementationSpecific
          backend:
...

thisispaulsmith commented 2 years ago

@akshaysngupta Sorry that was my mistake in the original post. The host is there, I just missed it from the sample.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    appgw.ingress.kubernetes.io/ssl-redirect: 'true'
    cert-manager.io/cluster-issuer: letsencrypt
    kubernetes.io/ingress.class: azure/application-gateway
spec:
  tls:
    - hosts:
        - testing.domain.com
      secretName: testing-tls
  rules:
    host: testing.domain.com
    http:
      paths:
        - path: /testpath
          pathType: ImplementationSpecific
          backend:
            service:
              name: test
              port:
                name: http
        - path: /testpath/*
          pathType: ImplementationSpecific
          backend:
            service:
              name: test
              port:
                name: http

jkroepke commented 1 year ago

could lets encrypt an issue here? because the secret for TLS will be created delayed, we have the same issue right now.

If

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    cert-manager.io/issue-temporary-certificate: 'true'

help to workaround this issue, it could be help.

anthonynguyen394 commented 1 year ago

I have the same issue, except that i am using the appgw.ingress.kubernetes.io/appgw-ssl-certificate annotation with a certificate imported from Keyvault. Here is my ingress definition:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: service-a
  namespace: default
  annotations:
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: <cert imported from keyvault>
    appgw.ingress.kubernetes.io/request-timeout: '60'
    appgw.ingress.kubernetes.io/ssl-redirect: 'true'
    kubernetes.io/ingress.class: azure/application-gateway
spec:
  rules:
    - host: teama.company.com
      http:
        paths:
          - path: /service-a
            pathType: Prefix
            backend:
              service:
                name: service-a
                port:
                  number: 80

I can see that this created a new listener on port 80 and a new rule attached to this listener, however, it doesn't use the Redirection to my https listener, which is what is said in the documentation, but rather just straight forwarding to the Backend pool. So it just ended up creating an unsecure http listener. See attachement:

I am using the following AGWIC image version: mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.3

housec84 commented 3 weeks ago

Same issue as @anthonynguyen394 - Anyone find a fix for this? Or are we stuck not being able to use AGIC if you want TLS and redirection?

Azure / application-gateway-kubernetes-ingress

SSL Redirect not applied when appgw.ingress.kubernetes.io/ssl-redirect: 'true' #1394