ingress_application_gateway block does not create new application gateway

[SaumyaBhushan]

Is there an existing issue for this?

• [X] I have searched the existing issues

Community Note

• Please vote on this issue by adding a :thumbsup: reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.4

AzureRM Provider Version

3.68.0

Affected Resource(s)/Data Source(s)

azurerm_kubernetes_cluster

Terraform Configuration Files

ingress_application_gateway {
    gateway_name = "gateway"
    subnet_id =  data.azurerm_subnet.ag_subnet.id
  }

Debug Output/Panic Output

Not getting any error while applying

Expected Behaviour

after adding this ingress_application_gateway block azurerm_kubernetes_cluster should enable AGIC addon and should create new application gateway with provided arguments details

Actual Behaviour

After successful run of terraform apply , aks is being created just fine with the add on enabled and attached application gateway but when I am clicking on gateway link it is showing resource not found . The message that is being displayed near ingress controller add on check box is Ingress controller has not created the application gateway yet. It may take up to 15 mins before the application gateway gets created. but I have waited for more than an hour and it has not been created yet .

Steps to Reproduce

terraform apply

Important Factoids

No response

References

No response

[lambertpan]

Also, when do import, it's not getting the 'subnet_cidr' and when do the terraform apply it says “ingress_application_gateway.0.subnet_id”: one of ingress_application_gateway.0.gateway_id,ingress_application_gateway.0.subnet_cidr,ingress_application_gateway.0.subnet_idmust be specified".

My existing aks was created using '2.93.0' and I am trying to use '3.64.0'. But unable to import the existing enabled ingress controller.

The image from imported backend

[neil-yechenwei]

Thanks for raising this issue. When ingressApplicationGateway.config.applicationGatewayName is set, service would automatically create the Application Gateway. But seems it failed to create the Application Gateway due to service API doesn't correctly assign the permission to the account that automatically deploys the Application Gateway. So filed an issue on https://github.com/Azure/azure-rest-api-specs/issues/25238 for tracking.

[SaumyaBhushan]

Thanks @neil-yechenwei

[gauravshegokar]

@SaumyaBhushan - were you able to find an alternative approach to this, I am seeing the same issue as you.

[SaumyaBhushan]

No not as of yet @gauravshegokar . But we can create new application gateway and add its id to aks terraform module by gateway_id

[johannesboyne]

@SaumyaBhushan thanks a lot for the suggestion, simply deploying another app-gateway and using the gateway_id! Even though it correctly links the gateway, we are currently stuck to get to a working version, e.g., binding to the correct ingress IPs and for example automating letsencrypt certificate pull etc. If you already deployed sth similar, it would be interesting to hear how you've done it.

[SaumyaBhushan]

Hey @johannesboyne ! Yes I have figured out a way in which we can bind the correct Ips . I can share on blog link here .

[johannesboyne]

Awesome, @SaumyaBhushan looking forward to it!

[SaumyaBhushan]

Hey @johannesboyne ! Here is the link to the blog that I was referring to . https://saumyapandey.hashnode.dev/aks-with-application-gateway-agic-using-terraform

[jtwaleson]

The original issue here can be fixed with this role assignment:

resource "azurerm_role_assignment" "aks_agic_integration" {
  scope = azurerm_virtual_network.example.id
  role_definition_name = "Network Contributor"
  principal_id = azurerm_kubernetes_cluster.example.ingress_application_gateway[0].ingress_application_gateway_identity[0].object_id
}

This will give the ingress-appgw-deployment enough privileges to bridge the gateway and the network that you're using.

I would suggest adding this to the documentation in the azurerm docs near ingress_application_gateway.

EDIT: note that initially the service will still give some errors, as we only give the role assignment after the service principal has been created "under the hood". After a couple of restarts it should fix itself as the permissions are applied.