This is an ingress controller that can be run on Azure Kubernetes Service (AKS) to allow an Azure Application Gateway to act as the ingress for an AKS cluster.
We can't see ingress load-balancer getting registered with AGIC after deployment to kubernetes cluster. Backend pool also did not get registered, No rules, backend settings created for our deployment with app gateway.
To Reproduce
Deploy your application with helm chart. It contains templates like Deployment, Service, Ingress etc.
Can't see listerner,backend pool, backend settings, rules gets registered with app gateway
Ingress Controller details
Output of kubectl describe pod <ingress controller> . The pod name can be obtained by running helm list.
Name: ingress-azure-xxx_xxxxx-q5fp2
Namespace: app-gateway-agic
Priority: 0
Node: aks-k8s1npde2-xxxx1147-vmssxxxx1i/XX.XXX.1.xxx
Start Time: Sat, 04 Jun 2022 02:01:29 +0530
Labels: app=ingress-azure
pod-template-hash=xxcxx6xxbxx
release=ingress-azure
Annotations: checksum/config: e8b0xxxxxxxxxxxxxxxxxxxxxxxxxx4e0ba91301e5b98bef77xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
kubectl.kubernetes.io/restartedAt: 2022-06-03T20:31:29Z
prometheus.io/port: 8123
prometheus.io/scrape: true
Status: Running
IP: XX.XXX.xX.XXX
IPs:
IP: XX.XXX.xx.XXX
Controlled By: ReplicaSet/ingress-azure-xxx_xxxxx
Containers:
ingress-azure:
Container ID: containerd://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Image: mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.1
Image ID: mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:cc131292df265926942e23ca5601a3de66e8feabcb81f705d8f7d84b740f81b6
Port:
Host Port:
State: Running
Started: Sat, 04 Jun 2022 02:01:30 +0530
Ready: True
Restart Count: 0
Liveness: http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
Readiness: http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
ingress-azure ConfigMap Optional: false
Environment:
AZURE_CLOUD_PROVIDER_LOCATION: /etc/appgw/azure.json
AGIC_POD_NAME: ingress-azure-xxx_xxxxx-q5fp2 (v1:metadata.name)
AGIC_POD_NAMESPACE: app-gateway-agic (v1:metadata.namespace)
AZURE_AUTH_LOCATION: /etc/Azure/Networking-AppGW/auth/armAuth.json
Mounts:
/etc/Azure/Networking-AppGW/auth from xxxxx-xxxxx-xxx-xxxx-service-principal-mount (ro)
/etc/appgw/ from azure (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xxxxxxx (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
azure:
Type: HostPath (bare host directory volume)
Path: /etc/kubernetes/
HostPathType: Directory
xxxxx-xxxxx-xxx-xxxx-service-principal-mount:
Type: Secret (a volume populated by a Secret)
SecretName: xxxxx-xxxxx-xxx-xxxx-service-principal
Optional: false
kube-api-access-6hgzv:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: xxxx
ConfigMapName: kubexxxx-xxx.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
Warning FailedApplyingAppGwConfig 2m9s (x4175 over 2d18h) azure/application-gateway network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/XXXXXX-XXXXXX-AKS-app-gateway/providers/Microsoft.Network/applicationGateways/XXXXXX-XXXXXX-AKS-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[]
Output of `kubectl logs .
I0606 14:54:15.086857 1 mutate_app_gateway.go:177] BEGIN AppGateway deployment
I0606 14:54:16.301639 1 mutate_app_gateway.go:183] END AppGateway deployment
I0606 14:54:16.301954 1 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"app-gateway-agic", Name:"ingress-azure-XXXXXXXXXXX-XXXXXXX", UID:"feee8a28-xxxx-xxxx-xxxxx-6c973e94e8b1", APIVersion:"v1", ResourceVersion:"246405881", FieldPath:""}): type: 'Warning' reason: 'FailedApplyingAppGwConfig' network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[]
E0606 14:54:16.302011 1 controller.go:141] network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[]
E0606 14:54:16.302023 1 worker.go:62] Error processing event.network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[
Any Azure support tickets associated with this issue.
Describe the bug Hi,
We can't see ingress load-balancer getting registered with AGIC after deployment to kubernetes cluster. Backend pool also did not get registered, No rules, backend settings created for our deployment with app gateway.
To Reproduce
Ingress Controller details
kubectl describe pod <ingress controller
> . Thehelm list
.Name: ingress-azure-xxx_xxxxx-q5fp2 Namespace: app-gateway-agic Priority: 0 Node: aks-k8s1npde2-xxxx1147-vmssxxxx1i/XX.XXX.1.xxx Start Time: Sat, 04 Jun 2022 02:01:29 +0530 Labels: app=ingress-azure pod-template-hash=xxcxx6xxbxx release=ingress-azure Annotations: checksum/config: e8b0xxxxxxxxxxxxxxxxxxxxxxxxxx4e0ba91301e5b98bef77xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx kubectl.kubernetes.io/restartedAt: 2022-06-03T20:31:29Z prometheus.io/port: 8123 prometheus.io/scrape: true Status: Running IP: XX.XXX.xX.XXX IPs: IP: XX.XXX.xx.XXX Controlled By: ReplicaSet/ingress-azure-xxx_xxxxx Containers: ingress-azure: Container ID: containerd://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Image: mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.1 Image ID: mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:cc131292df265926942e23ca5601a3de66e8feabcb81f705d8f7d84b740f81b6 Port:
Host Port:
State: Running
Started: Sat, 04 Jun 2022 02:01:30 +0530
Ready: True
Restart Count: 0
Liveness: http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
Readiness: http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
ingress-azure ConfigMap Optional: false
Environment:
AZURE_CLOUD_PROVIDER_LOCATION: /etc/appgw/azure.json
AGIC_POD_NAME: ingress-azure-xxx_xxxxx-q5fp2 (v1:metadata.name)
AGIC_POD_NAMESPACE: app-gateway-agic (v1:metadata.namespace)
AZURE_AUTH_LOCATION: /etc/Azure/Networking-AppGW/auth/armAuth.json
Mounts:
/etc/Azure/Networking-AppGW/auth from xxxxx-xxxxx-xxx-xxxx-service-principal-mount (ro)
/etc/appgw/ from azure (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xxxxxxx (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
azure:
Type: HostPath (bare host directory volume)
Path: /etc/kubernetes/
HostPathType: Directory
xxxxx-xxxxx-xxx-xxxx-service-principal-mount:
Type: Secret (a volume populated by a Secret)
SecretName: xxxxx-xxxxx-xxx-xxxx-service-principal
Optional: false
kube-api-access-6hgzv:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: xxxx
ConfigMapName: kubexxxx-xxx.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
Warning FailedApplyingAppGwConfig 2m9s (x4175 over 2d18h) azure/application-gateway network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/XXXXXX-XXXXXX-AKS-app-gateway/providers/Microsoft.Network/applicationGateways/XXXXXX-XXXXXX-AKS-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[]
I0606 14:54:15.086857 1 mutate_app_gateway.go:177] BEGIN AppGateway deployment I0606 14:54:16.301639 1 mutate_app_gateway.go:183] END AppGateway deployment I0606 14:54:16.301954 1 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"app-gateway-agic", Name:"ingress-azure-XXXXXXXXXXX-XXXXXXX", UID:"feee8a28-xxxx-xxxx-xxxxx-6c973e94e8b1", APIVersion:"v1", ResourceVersion:"246405881", FieldPath:""}): type: 'Warning' reason: 'FailedApplyingAppGwConfig' network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[] E0606 14:54:16.302011 1 controller.go:141] network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[] E0606 14:54:16.302023 1 worker.go:62] Error processing event.network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="ApplicationGatewayPortNotValidForProtocol" Message="Port 443 is not valid for protocol Http in httpListeners /subscriptions/-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/xxxx-XXXXXX-XXX-app-gateway/providers/Microsoft.Network/applicationGateways/xxxx-XXXXXX-XXX-app-gateway/httpListeners/fl-77d00c3f4e69d3f2c9f38a0059a5ac21." Details=[