search

Azure / AKS-Construction

Accelerate your onboarding to AKS with; Helper Web App, bicep templating and CI/CD samples. Flexible & secure AKS baseline implementations in a Microsoft + community maintained reference implementation.
https://azure.github.io/AKS-Construction/
MIT License
356 stars 165 forks source link

Application Gateway and AKS nodes created in separate resource groups - Application Gateway Insufficient Permission on Subnet #679

Closed saikovvuri closed 6 months ago

saikovvuri commented 8 months ago

Describe the bug The deployment template creates App Gateway instance and AKS nodes in separate resource group resulting in AGIC identity not having the network contributor permission on the App Gateway subnet.

To Reproduce Steps to reproduce the behavior: Ran this deployment script generated by the helper

az group create -l EastUS2 -n az-k8s-aauj-rg

az deployment group create -g az-k8s-aauj-rg --template-uri https://github.com/Azure/AKS-Construction/releases/download/0.10.3/main.json --parameters \ resourceName=az-k8s-aauj \ agentCount=1 \ upgradeChannel=stable \ JustUseSystemPool=true \ agentCountMax=20 \ osDiskType=Managed \ osDiskSizeGB=32 \ custom_vnet=true \ registries_sku=Premium \ acrPushRolePrincipalId=$(az ad signed-in-user show --query id --out tsv) \ omsagent=true \ retentionInDays=30 \ networkPolicy=azure \ azurepolicy=audit \ authorizedIPRanges="[\"172.5.172.116/32\"]" \ ingressApplicationGateway=true \ appGWcount=0 \ appGWsku=Standard_v2 \ appGWmaxCount=10 \ appgwKVIntegration=true \ keyVaultAksCSI=true \ keyVaultCreate=true \ keyVaultOfficerRolePrincipalId=$(az ad signed-in-user show --query id --out tsv) \ automationAccountScheduledStartStop=Weekday

Got this error from the agic pod logs controller.go:141] network.ApplicationGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status= Code="ApplicationGatewayInsufficientPermissionOnSubnet"

Expected behavior Resolve this by following this instructions

Get application gateway id from AKS addon profile appGatewayId=$(az aks show -n myCluster -g myResourceGroup -o tsv --query "addonProfiles.ingressApplicationGateway.config.effectiveApplicationGatewayId")

Get Application Gateway subnet id appGatewaySubnetId=$(az network application-gateway show --ids $appGatewayId -o tsv --query "gatewayIPConfigurations[0].subnet.id")

Get AGIC addon identity agicAddonIdentity=$(az aks show -n myCluster -g myResourceGroup -o tsv --query "addonProfiles.ingressApplicationGateway.identity.clientId")

Assign network contributor role to AGIC addon identity to subnet that contains the Application Gateway az role assignment create --assignee $agicAddonIdentity --scope $appGatewaySubnetId --role "Network Contributor"

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

github-actions[bot] commented 7 months ago

Issue smells stale, no activity for 30 days. Stale Label will be removed if the issue is updated, otherwise closed in a month.