Please add reconcilePeriodSeconds as an option for cluster add-on

[03eltond]

Currently, there is an issue where redeploying bicep (probably ARM too?) templates for AG removes the backend configuration set by AGIC (even if nothing has changed): https://github.com/Azure/bicep/issues/2316

As a workaround, reconcilePeriodSeconds can be used to force AGIC to sync to AG: https://azure.github.io/application-gateway-kubernetes-ingress/features/agic-reconcile/

While this workaround is rather unfortunate due to implied downtime of 30 seconds or more, it is at least a solution. However, this workaround is not available for AGIC when configured as an AKS addon: https://docs.microsoft.com/en-us/azure/architecture/example-scenario/aks-agic/aks-agic#alternatives

Installing AGIC as a helm chart instead of a cluster addon isn't a deal breaker, but it sure is convenient to take advantage of the declarative nature of defining it as an add-on for our IaC (which further enables us to declaratively set up RBAC between the AGIC Identity and the resource group, subnet, and AG).

Solutions I would prefer, in order:

• Any solution which doesn't cause any down time. This limitation prevents us from deploying during most hours except for the case of emergencies. This could maybe be achieved by allowing the backend configurations to be optional in the bicep templates (see #2316 above), such that there is no sync performed if omitted.
• Just do the reconcile automatically, so I don't have to configure this.
• Expose the reconcilePeriodSeconds as something I can configure as an add-on.

[f-vt]

Hi,

I'm quite new on Kubernetes/AKS stuff, so feel free to correct me if I made mistakes.

I encounter exactly same issues, deploying infrastructure with ARM templates including the Application Gateway then used as AGIC with AKS:

• Due to the fact that App Gw template must contain all stuff like listeners, backend pools, ...
• I'm setting dumb ones in the ARM template.
• Then when redeploying it, it obviously remove all the setup made by AGIC, causing downtime.

AGIC AKS plugin doesn't seems to have Reconcile period enabled and as said setting this through regular ingress YAML deployment is not supported yet in this plugin... So AGIC AKS plugin never correct the deployment, you have to redeploy the ingress to update the application gateway... Not great.

I agree that reconcilePeriodSeconds is the least preferable option, as it will still cause downtime, and also often fails the ARM Template deployment because of conflicts with the reconcile update tasks on the application Gateway. I wish that we could deploy an Application gateway "Empty"... (vote here / original github issue closed by MS)

However, for those who wants to make Reconcile works with the AKS AGIC plugin as we don't have better option at the time being:

• Looking at the helm templates and the AGIC code on github, it seems that helm setting reconcilePeriodSeconds is in the end an environment variable passed to the agic containers : agic environment.go code

• Microsoft uses ingress-appgw-deployment deployment template and ingress-appgw-cm configMap when using the plugin on AKS.

• Looking at the deployment we see that it uses the same image than the Helm one, so that it should take the environment variable in account if set. Environment variables are not taken from the configMap so we don't car about the configMap here.

Then you can just add the environment variable to the existing deployment with kubectl set env. I run these commands juste after my ApplicationGateway then AKS (with AGIC plugin set) deployment to enable the feature:

kubectl set env deployment/ingress-appgw-deployment -n kube-system --list
kubectl set env deployment/ingress-appgw-deployment -n kube-system --all RECONCILE_PERIOD_SECONDS=30
kubectl rollout restart deployment/ingress-appgw-deployment -n kube-system
kubectl rollout status deployment/ingress-appgw-deployment -n kube-system
kubectl set env deployment/ingress-appgw-deployment -n kube-system --list

Hope this helps a bit.

NB; In case it helps other setting up their preferred method: I first had in mind to setup a Kubernetes cron job with an image containing kubectl binary to run periodically the kubectl rollout restart deployment/ingress-appgw-deployment -n kube-system command, but as I figured out how to enable the Reconcile setting then it was not needed.

[f-vt]

After some testing, as Application Gateway ARM update task takes around 2 to 4 minutes to complete, I recommend not setting less than 270 seconds for the RECONCILE_PERIOD_SECONDS (accepted values from 30 to 300). This will let enough time to the ARM template deployment to update the Application Gateway configuration before re-correcting it.

This should avoid you getting "Conflict" error in the ARM deployment task for the Application Gateway.

[scorpionfly7]

Is there any updates, we need this feature!!! Why there exists AGiC app gateway, if it not useful?