Deployment fails when using Custom Networking and when using custom Address space

[romerve]

Describe the bug When using AKS Construction helper, the deployment fails with the following error:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"NetcfgInvalidSubnet\\\",\\r\\n    \\\"message\\\": \\\"Subnet 'AzureFirewallManagementSubnet' is not valid in virtual network 'vnet-aks-rv-test'.\\\",\\r\\n    \\\"details\\\": []\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

It appears that even when no Azure Firewall is requested in the helper, the helper attempts to create an AzureFirewallManagementSubnet

To Reproduce Steps to reproduce the behavior:

1. Go to Networking Details tab
2. Under Network plugin, select "Use CNI..."
3. Under Default or Custom VNET, select Custom Networking
4. VNET Address space, anything, but the default space. EX: 10.200.0.0/16
5. AKS Nodes Subnet, anything within the range. EX: 10.200.0.0/20

Alternatively, it can be recreated it by: URL for AKS-Construction helper config

# Create Resource Group
az group create -l EastUS2 -n aks-rv-test-rg

# Deploy template with in-line parameters
az deployment group create -g aks-rv-test-rg  --template-uri https://github.com/Azure/AKS-Construction/releases/download/0.9.5/main.json --parameters \
    resourceName=aks-rv-test \
    kubernetesVersion=1.25.2 \
    upgradeChannel=stable \
    SystemPoolType=Standard \
    nodePoolName=npsymuser01 \
    agentCountMax=20 \
    osDiskType=Managed \
    custom_vnet=true \
    vnetAddressPrefix=10.200.0.0/19 \
    vnetAksSubnetAddressPrefix=10.200.0.0/21 \
    enable_aad=true \
    AksDisableLocalAccounts=true \
    enableAzureRBAC=true \
    adminPrincipalId=$(az ad signed-in-user show --query id --out tsv) \
    registries_sku=Standard \
    acrPushRolePrincipalId=$(az ad signed-in-user show --query id --out tsv) \
    enableTelemetry=false \
    omsagent=true \
    retentionInDays=30 \
    fileCSIDriver=false \
    diskCSIDriver=false \
    keyVaultAksCSI=true \
    keyVaultCreate=true \
    keyVaultOfficerRolePrincipalId=$(az ad signed-in-user show --query id --out tsv) \
    keyVaultAksCSIPollInterval=5m \
    fluxGitOpsAddon=true

Expected behavior Deployment succeeds, and provisioned VNet does not have an AzureFirewallManagementSubnet created without requesting it.

[Gordonby]

I think this could have been fixed in #480 and just needs released. @MattLeach25 - thoughts?

[MattLeach25]

I think this could have been fixed in #480 and just needs released. @MattLeach25 - thoughts?

Yeah this is fixed with the new release.

[Gordonby]

We'll aim to cut a release tomorrow. @romerve - if you clone the repo and target bicep/main.bicep instead of the release - then you should be good.

[Gordonby]

@MattLeach25 created a release today, so this should be fixed.