search

Azure / application-gateway-kubernetes-ingress

This is an ingress controller that can be run on Azure Kubernetes Service (AKS) to allow an Azure Application Gateway to act as the ingress for an AKS cluster.
https://azure.github.io/application-gateway-kubernetes-ingress
MIT License
673 stars 419 forks source link

Reconciliation is always triggering ARM deployment #885

Open besteban1989 opened 4 years ago

besteban1989 commented 4 years ago

When you deploy AGIC with reconciliation enabled, let's say each minute.

Even if you don't have a change you always see a "Updating message" in the Application Gateway in Azure.

akshaysngupta commented 4 years ago

@besteban1989 Can you help me repro this locally by sharing your ingress/deployment spec ?

besteban1989 commented 4 years ago

Sure this is the ingress object: 

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/ingress.class: azure/application-gateway
    meta.helm.sh/release-name: customer-api
    meta.helm.sh/release-namespace: enterprise-services
  creationTimestamp: "2020-05-21T13:34:34Z"
  generation: 1
  labels:
    app: customer-api
    app.kubernetes.io/managed-by: Helm
    chart: customer-api-34555
    heritage: Helm
    release: customer-api
  name: customer-api
  namespace: enterprise-services
  resourceVersion: "2327278"
  selfLink: /apis/extensions/v1beta1/namespaces/enterprise-services/ingresses/customer-api
  uid: 112f76ad-e5b7-43cf-b074-fd17691a0a2d
spec:
  rules:
  - host: customer-api.test-url.com
    http:
      paths:
      - backend:
          serviceName: customer-api
          servicePort: http
        path: /
  tls:
  - hosts:
    - customer-api.test-url.com
    secretName: tls-secret-customer-api
status:
  loadBalancer:
    ingress:
    - ip: someip

The service: 

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: customer-api
    meta.helm.sh/release-namespace: enterprise-services
  creationTimestamp: "2020-05-27T16:50:05Z"
  labels:
    app: customer-api
    app.kubernetes.io/managed-by: Helm
    chart: customer-api-34555
    heritage: Helm
    release: customer-api
  name: customer-api
  namespace: enterprise-services
  resourceVersion: "2327267"
  selfLink: /api/v1/namespaces/enterprise-services/services/customer-api
  uid: ea5c3eaf-b3ef-4ed7-98b9-6494bbfc7817
spec:
  clusterIP: 10.0.203.55
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
  selector:
    app: customer-api
    release: customer-api
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

The Deployment: 

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
    meta.helm.sh/release-name: customer-api
    meta.helm.sh/release-namespace: enterprise-services
  creationTimestamp: "2020-05-27T16:50:05Z"
  generation: 1
  labels:
    app: customer-api
    app.kubernetes.io/managed-by: Helm
    chart: customer-api-34555
    heritage: Helm
    release: customer-api
  name: customer-api
  namespace: enterprise-services
  resourceVersion: "2327818"
  selfLink: /apis/apps/v1/namespaces/enterprise-services/deployments/customer-api
  uid: 79938009-8642-4507-8d0d-676754a22803
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: customer-api
      release: customer-api
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        aadpodidbinding: aadpodkeyvault
        app: customer-api
        release: customer-api
    spec:
      containers:
        image: secimages.azurecr.io/customer-api:34555
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /health/live
            port: 80
            scheme: HTTP
          initialDelaySeconds: 90
          periodSeconds: 90
          successThreshold: 1
          timeoutSeconds: 1
        name: customer-api
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /health/ready
            port: 80
            scheme: HTTP
          initialDelaySeconds: 90
          periodSeconds: 90
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: regcred
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2020-05-27T16:52:15Z"
    lastUpdateTime: "2020-05-27T16:52:15Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2020-05-27T16:50:05Z"
    lastUpdateTime: "2020-05-27T16:52:15Z"
    message: ReplicaSet "customer-api-585cdf9996" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1
besteban1989 commented 4 years ago

Also I attach the logs.

Regards.

El mié., 27 may. 2020 a las 13:54, Akshay Gupta (notifications@github.com) escribió:

@besteban1989 https://github.com/besteban1989 Can you help me repro this locally by sharing your ingress/deployment spec ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Azure/application-gateway-kubernetes-ingress/issues/885#issuecomment-634905603, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZIAJO6ZXALH76WYGBFDTDRTVVXPANCNFSM4NMJNYYQ .

--

Show log entriesMatch expression Filter Loading...ERROR: logging before flag.Parse: I0527 23:26:17.819059 1 utils.go:111] Using verbosity level 5 from environment variable APPGW_VERBOSITY_LEVEL I0527 23:26:17.854127 1 environment.go:230] KUBERNETES_WATCHNAMESPACE is not set. Watching all available namespaces. I0527 23:26:17.854236 1 main.go:125] App Gateway Details: Subscription: d299c2ef-b5ab-4281-9e17-61d7b5534836, Resource Group: staging-infra-rg, Name: staging-infra-appgw I0527 23:26:17.854336 1 auth.go:46] Creating authorizer from Azure Managed Service Identity I0527 23:26:17.854256 1 httpserver.go:57] Starting API Server on :8123 I0527 23:26:19.422027 1 main.go:168] Ingress Controller will observe all namespaces. I0527 23:26:19.732623 1 main.go:196] Unable to associate Application Gateway subnet '/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/my-network-rg/providers/Microsoft.Network/virtualNetworks/staging-vnet/subnets/staging-app-gateway-subnet' with route table '/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/MC_staging-infra-rg_staging-infra-aks_eastus2/providers/Microsoft.Network/routeTables/aks-agentpool-29123037-routetable' due to error (this is relevant for AKS clusters using 'Kubenet' network plugin): [network.RouteTablesClient#Get: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '2cbef673-3bdc-403e-81ff-3379bf5d6a21' with object id '2cbef673-3bdc-403e-81ff-3379bf5d6a21' does not have authorization to perform action 'Microsoft.Network/routeTables/read' over scope '/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/MC_staging-infra-rg_staging-infra-aks_eastus2/providers/Microsoft.Network/routeTables/aks-agentpool-29123037-routetable' or the scope is invalid. If access was recently granted, please refresh your credentials."] I0527 23:26:19.732714 1 context.go:132] k8s context run started I0527 23:26:19.732750 1 context.go:176] Waiting for initial cache sync I0527 23:26:20.081764 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-my-queue-service] I0527 23:26:20.086832 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-web-builder] I0527 23:26:20.091920 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-ui-web-builder] I0527 23:26:20.096933 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-app-repository] I0527 23:26:20.101989 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-rbac-service] I0527 23:26:20.106764 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-my-portal] I0527 23:26:20.111685 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-file-reader] I0527 23:26:20.116254 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-customer-api] I0527 23:26:20.120450 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-console-service] I0527 23:26:20.124587 1 secretstore.go:152] Converted secret [enterprise-services/tls-secret-project-repository-service] I0527 23:26:20.132905 1 context.go:189] Initial cache sync done I0527 23:26:20.132920 1 context.go:190] k8s context run finished I0527 23:26:20.133012 1 controller.go:157] Reconciler Ticker task started with period: 300 I0527 23:26:20.133032 1 worker.go:39] Worker started I0527 23:26:20.274671 1 mutate_aks.go:90] [mutate_aks] Found IPs: map[/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP:52.177.242.143] I0527 23:26:20.274829 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.281735 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/app-repository I0527 23:26:20.281752 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/app-repository IP to 52.177.242.143 I0527 23:26:20.281759 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.292231 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-file-reader I0527 23:26:20.292317 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-file-reader IP to 52.177.242.143 I0527 23:26:20.292333 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.295955 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-customer-api I0527 23:26:20.295969 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-customer-api IP to 52.177.242.143 I0527 23:26:20.295976 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.300223 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-console-service I0527 23:26:20.300332 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-console-service IP to 52.177.242.143 I0527 23:26:20.300367 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.304539 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-project-repository-service I0527 23:26:20.304555 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-project-repository-service IP to 52.177.242.143 I0527 23:26:20.304562 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.307844 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-rbac-service I0527 23:26:20.307861 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-rbac-service IP to 52.177.242.143 I0527 23:26:20.307868 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.311332 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/core-my-queue-service I0527 23:26:20.311412 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/core-my-queue-service IP to 52.177.242.143 I0527 23:26:20.311422 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.316637 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/my-portal I0527 23:26:20.316921 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/my-portal IP to 52.177.242.143 I0527 23:26:20.316955 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.321270 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/orchestration-web-builder I0527 23:26:20.321356 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/orchestration-web-builder IP to 52.177.242.143 I0527 23:26:20.321365 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.331590 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/ui-web-builder I0527 23:26:20.331687 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/ui-web-builder IP to 52.177.242.143 I0527 23:26:20.331738 1 mutate_aks.go:65] [mutate_aks] Resolving IP for ID (/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP) I0527 23:26:20.479735 1 context.go:504] IP 52.177.242.143 already set on Ingress enterprise-services/web-builder I0527 23:26:20.479756 1 mutate_aks.go:72] [mutate_aks] Updated Ingress enterprise-services/web-builder IP to 52.177.242.143 I0527 23:26:20.484881 1 mutate_app_gateway.go:70] Existing App Gateway config: { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw", -- Existing App Gwy Config -- "location": "eastus2", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddressPools": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/defaultaddresspool", -- Existing App Gwy Config -- "name": "defaultaddresspool", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-app-repository-proxy-bp-5001", -- Existing App Gwy Config -- "name": "pool-enterprise-services-app-repository-proxy-bp-5001", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.8" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-file-reader-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-file-reader-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.18" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-customer-api-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-customer-api-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.80" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-console-service-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-console-service-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.120" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-project-repository-service-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-project-repository-service-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.19" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-rbac-service-proxy-bp-5001", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-rbac-service-proxy-bp-5001", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.51" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-core-my-queue-service-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-core-my-queue-service-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.69" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-my-portal-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-my-portal-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.31" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-orchestration-web-builder-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-orchestration-web-builder-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.89" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-ui-web-builder-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-ui-web-builder-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.150" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendAddressPools/pool-enterprise-services-web-builder-http-bp-80", -- Existing App Gwy Config -- "name": "pool-enterprise-services-web-builder-http-bp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "backendAddresses": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "ipAddress": "10.142.65.100" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "backendHttpSettingsCollection": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-app-repository-serv-d61b3e26bb801930e5cd40903ff88015", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-app-repository-serv-d61b3e26bb801930e5cd40903ff88015", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 5001, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-app-repository-serv-3f3db6f4de1ea1e20f08977e0399ffcf" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-file-reader-http-80-core-file-reader", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-file-reader-http-80-core-file-reader", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-file-reader-http-core-file-reader" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-customer-api-http-80-core-customer-api", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-customer-api-http-80-core-customer-api", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-customer-api-http-core-customer-api" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-console-service-http-80-core-console-service", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-console-service-http-80-core-console-service", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-console-service-http-core-console-service" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-project-repository--7636272850c33c53c0bb4dc6fd4e4819", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-project-repository--7636272850c33c53c0bb4dc6fd4e4819", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-project-repository--eb48af6669c49079c824b6ea041996fd" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-rbac-service-proxy-5001-core-rbac-service", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-rbac-service-proxy-5001-core-rbac-service", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 5001, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-rbac-service-proxy-core-rbac-service" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-core-timed-wait-queue-se-cb9485f46715a475116858a4d376ab4f", -- Existing App Gwy Config -- "name": "bp-enterprise-services-core-timed-wait-queue-se-cb9485f46715a475116858a4d376ab4f", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-core-timed-wait-queue-se-e47cfcd58663e8e9b1e6eeb976dfcd18" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-my-portal-http-80-my-portal", -- Existing App Gwy Config -- "name": "bp-enterprise-services-my-portal-http-80-my-portal", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-my-portal-http-my-portal" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-orchestration-web-manife-ff9cddcd19821f77a710d6278f6a7ef5", -- Existing App Gwy Config -- "name": "bp-enterprise-services-orchestration-web-manife-ff9cddcd19821f77a710d6278f6a7ef5", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-orchestration-web-manife-4b5bc54053bdead38f4bfb7db0a8b412" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-ui-web-builder-http-80-ui-web-builder", -- Existing App Gwy Config -- "name": "bp-enterprise-services-ui-web-builder-http-80-ui-web-builder", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-ui-web-builder-http-ui-web-builder" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/bp-enterprise-services-web-builder-http-80-web-builder", -- Existing App Gwy Config -- "name": "bp-enterprise-services-web-builder-http-80-web-builder", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/pb-enterprise-services-web-builder-http-web-builder" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/backendHttpSettingsCollection/defaulthttpsetting", -- Existing App Gwy Config -- "name": "defaulthttpsetting", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "cookieBasedAffinity": "Disabled", -- Existing App Gwy Config -- "pickHostNameFromBackendAddress": false, -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "probe": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/defaultprobe-Http" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requestTimeout": 30 -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "enableHttp2": false, -- Existing App Gwy Config -- "frontendIPConfigurations": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP", -- Existing App Gwy Config -- "name": "appGatewayFrontendIP", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "privateIPAllocationMethod": "Dynamic", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "publicIPAddress": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/publicIPAddresses/staging-infra-pip" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "frontendPorts": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/fp-80", -- Existing App Gwy Config -- "name": "fp-80", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "port": 80, -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort", -- Existing App Gwy Config -- "name": "httpsPort", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "port": 443, -- Existing App Gwy Config -- "provisioningState": "Succeeded" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "gatewayIPConfigurations": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/gatewayIPConfigurations/appGatewayIpConfig", -- Existing App Gwy Config -- "name": "appGatewayIpConfig", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "subnet": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/my-network-rg/providers/Microsoft.Network/virtualNetworks/staging-vnet/subnets/staging-app-gateway-subnet" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "httpListeners": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-0117f72781b070f48462fc9573bb6545", -- Existing App Gwy Config -- "name": "fl-0117f72781b070f48462fc9573bb6545", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "my-queue-service.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-my-queue-service" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-1543ed7514da1b447e3cdb44b32db3dc", -- Existing App Gwy Config -- "name": "fl-1543ed7514da1b447e3cdb44b32db3dc", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "customer-api.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-customer-api" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-34a580b2ed834cd91b3c838983e82c44", -- Existing App Gwy Config -- "name": "fl-34a580b2ed834cd91b3c838983e82c44", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "my-portal.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-my-portal" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-444f8f0d363d1243399e9b7818f037c6", -- Existing App Gwy Config -- "name": "fl-444f8f0d363d1243399e9b7818f037c6", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "web-builder.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-web-builder" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-742eea1c10aba55318986e4e19035ac6", -- Existing App Gwy Config -- "name": "fl-742eea1c10aba55318986e4e19035ac6", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/fp-80" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "project-repository-service.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-83d3cf91d2701df4b39b29f5fcd532c6", -- Existing App Gwy Config -- "name": "fl-83d3cf91d2701df4b39b29f5fcd532c6", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/fp-80" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "file-reader.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-a03add58a392c43261e7147cd3139fdd", -- Existing App Gwy Config -- "name": "fl-a03add58a392c43261e7147cd3139fdd", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "rbac-service.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-rbac-service" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-a0a34241fb0660664415ca805b727dd6", -- Existing App Gwy Config -- "name": "fl-a0a34241fb0660664415ca805b727dd6", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "app-repository.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-app-repository" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-b3ee37c27c2a9f1579159b1ddd6629ed", -- Existing App Gwy Config -- "name": "fl-b3ee37c27c2a9f1579159b1ddd6629ed", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "ui-web-builder.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-ui-web-builder" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/httpListeners/fl-f3a072e7df09eb8b9fa99664f2141944", -- Existing App Gwy Config -- "name": "fl-f3a072e7df09eb8b9fa99664f2141944", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "frontendIPConfiguration": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendIPConfigurations/appGatewayFrontendIP" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "frontendPort": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/frontendPorts/httpsPort" -- Existing App Gwy Config -- }, -- Existing App Gwy Config -- "hostnames": [ -- Existing App Gwy Config -- "console-service.test-url.com" -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "protocol": "Https", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config -- "requireServerNameIndication": false, -- Existing App Gwy Config -- "sslCertificate": { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/sslCertificates/enterprise-services-tls-secret-console-service" -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- } -- Existing App Gwy Config -- ], -- Existing App Gwy Config -- "operationalState": "Running", -- Existing App Gwy Config -- "probes": [ -- Existing App Gwy Config -- { -- Existing App Gwy Config -- "id": "/subscriptions/d299c2ef-b5ab-4281-9e17-61d7b5534836/resourceGroups/staging-infra-rg/providers/Microsoft.Network/applicationGateways/staging-infra-appgw/probes/defaultprobe-Http", -- Existing App Gwy Config -- "name": "defaultprobe-Http", -- Existing App Gwy Config -- "properties": { -- Existing App Gwy Config -- "host": "localhost", -- Existing App Gwy Config -- "interval": 30, -- Existing App Gwy Config -- "match": {}, -- Existing App Gwy Config -- "minServers": 0, -- Existing App Gwy Config -- "path": "/", -- Existing App Gwy Config -- "pickHostNameFromBackendHttpSettings": false, -- Existing App Gwy Config -- "protocol": "Http", -- Existing App Gwy Config -- "provisioningState": "Succeeded", -- Existing App Gwy Config --

marcinkoperski commented 3 years ago

Hi. We've noticed that every such ARM deployment terminates already established websocket TCP sessions (don't know if regular HTTP sessions are also affected). We had to disable reconciliation because of that.

KodWhat commented 3 years ago

Hi, To add to this issue, the reconcile operation seems to trigger an ARM deploy that takes priority over other deployments. This causes any Azure pipeline that run an ARM template deployment on the App Gateway to be cancelled with the following message in Azure : 

{
    "status": "Canceled",
    "error": {
        "code": "Canceled",
        "message": "Operation was canceled.",
        "details": [
            {
                "code": "CanceledAndSupersededDueToAnotherOperation",
                "message": "Operation PutApplicationGatewayOperation (7b3831b7-3634-425d-a6c6-ac941b8a587b) was canceled and superseded by operation PutApplicationGatewayOperation (f9f24c01-afcf-496e-9b4f-e711f6690afb)."
            }
        ]
    }
}

This renders the reconcile feature quite useless as the main use of it is to rebuild the rules after an ARM update.

I would suggest to offer a way to ask AGIC to reconcile manually

jbaarssen commented 3 years ago

Hi, We are experiencing the same issues with the reconcile operation. Is there an other solution to deal with ARM deployments from the infrastructure-as-code and the AGIC pod? Because the deployments are indeed cancelled whenever AGIC is deploying changes. This causes the problem that we sometimes have to disable the ingress controller to make certain changes to our gateways.

f-vt commented 11 months ago

Hi,

I'm trying Reconcile with AGIC (using AKS AGIC plugin #1446 so not the manual Helm deployment, but it's using the same code) and I do not face this issue. On my testing environment the Application Gateway update by ingressapplicationgateway-... is only triggered when something change on the App Gateway configuration, or if the ingress configuration has changed, or if the ingress pods were redeployed, or if the service/target application deployment/pod has changed.

However I had to set the reconcile period not lower than 270 seconds to not get conflicts when deploying ARM template.

I also suggest to vote for Microsoft to allow us to deploy "empty" Application Gateway so this would solves most of the Infrastructure as Code issues we face when using AGIC and IAC for application gateways deployments.. vote here / original github issue closed by MS