bicep what-if says existing subnet will be updated when importing existing vnet provisioned using ARM

[mush-monster]

I have an existing vnet with a bunch of subnets that were provisioned and managed using ARM templates.

We have a need to add additional subnets to the vnet. So, we have decided to move towards bicep and have created a template that would match the config of the existing vnet and subnets plus the new ones we want to add.

For example, we have an existing vnet with subnet a, b, c which was provisioned using ARM templates. I have now created a bicep template which matched the existing vnet and the existing subnets and have also added new subnets lets say d and e.

when I do a bicep what-if, i was expecting to see that the the output would say that subnet d and e would be added and no changes to either vnet or existing subnets.

However, what I see is, no changes to vnet config but subnet a is getting updated to reflect subnet d config, subnet b is getting updated to reflect subnet e config and subnet a, b, c are getting created as new.

[mush-monster]

Also, something that I have noticed is, if i update the template to just include the existing subnets and remove the new subnets then bicep what-if says no changes.

Please note that this is the first time bicep templates are being run on that resource group and until now we have been using arm templates to manage the resource.

Also, subnet config is passed as a object array to the vnet resource.

[alex-frankel]

It would be helpful to see the full what-if response. I suspect the issue is related to the order the subnets are declared in the bicep file. Unfortunately, the order returned by the Network RP is not deterministic and we don't have a primary key to sort the list, so what-if ends up comparing the wrong subnets.

[alex-frankel]

This GH issue still needs a repro, but at least one issue is that the order of the subnets declared in the bicep file/ARM template is different than the order of subnets returned by the Networking Resource Provider (RP) on a GET request for the virtual network. This means that when the two sources (the bicep side and the GET response side) are compared, they get compared incorrectly. Instead of new subnet A being compared to current subnet A, we compare new subnet A to current subnet B and so on.

The fix for the issue is to normalize the order of subnets based on a primary key designated by the Networking RP. This would need to happen in the what-if API, but as of now we do not have the resourcing to fix the issue properly. In the meantime, you may be able to remove the noise themselves by sorting the subnets in the bicep template alphabetically by the “name” property. This is likely to be error prone, but if the issue is causing enough pain, it may be worth trying.

[dima-b]

I have the same issue and unfortunately rearranging of subnets in bicep file doesn't help. I need to add a new subnet and instead of adding just one new subnet it starts to replace existing subnets. For example I have subnet with names "B", "C", "D", "E" and I need to add a new subnet with name AzureFirewallSubnet. once I run what-if it displays that the following changes would occur B=>AzureFirewallSubnet C=>B D=>C +new subnet with "D" will be created it doesn't matter where you place this new AzureFirewallSubnet in bicep code it always displays changes above. The only way to fix the issue is to rename AzureFirewallSubnet to "E" or any other symbol following after D, but unfortunately it is not possible as this subnet must be named exactly AzureFirewallSubnet. Another workaround is to create this subnet manually via Portal.

[lansalot]

This is especially jarring with the MS-provided Data Landing Zone templates... Sample section of WhatIf output:

     ~ 6:
        - properties.serviceEndpoints: [
            0:
              service: "Microsoft.Storage"
          ]
        ~ name:                     "DataIntegration001Subnet" => "DataProduct001Subnet"
        ~ properties.addressPrefix: "10.4.6.0/24" => "10.4.8.0/24"
      ~ 7:
        - properties.serviceEndpoints: [
            0:
              service: "Microsoft.Storage"
          ]
        ~ name:                     "DataIntegration002Subnet" => "DataProduct002Subnet"
        ~ properties.addressPrefix: "10.4.7.0/24" => "10.4.9.0/24"
      ~ 8:
        - properties.serviceEndpoints: [
            0:
              service: "Microsoft.Storage"
          ]
        + properties.delegations: [
            0:
              name:                   "PowerBIGatewaySubnetDelegation"
              properties.serviceName: "Microsoft.PowerPlatform/vnetaccesslinks"
          ]
        ~ name:                                         "DataProduct001Subnet" => "PowerBIGatewaySubnet"
        ~ properties.addressPrefix:                     "10.4.8.0/24" => "10.4.5.0/24"
        ~ properties.privateEndpointNetworkPolicies:    "Disabled" => "Enabled"
        ~ properties.privateLinkServiceNetworkPolicies: "Disabled" => "Enabled"

The ARM template for data landing zone provisions them as an array of subnets, as seen here at the following link: https://github.com/Azure/data-landing-zone/blob/main/infra/modules/network.bicep

It can make reading the WhatIf output time consuming as you end up looking for what's really changing, rather than all this that shouldn't be (and actually won't be) changing.

@alex-frankel To test, you can deploy that BICEP template, then just WhatIf it again. Subnets will threaten to change down to ordering in the whatif output. Of course, if actually deployed, they remain untouched as they're correct.

If requiring a PK to validate on, would using the addressPrefix not be ideal? The name isn't changing either of course.

[shivakumarg06]

Hello,

I am encountering a similar issue with the Infrastructure as Code (IaC) deployment using Template Specs and ARM Templates. When I try to add additional subnets to the subnets, it throws the error due to the subnet being already allocated to a resource or due to the subnet delegation,

As a result, I am receiving errors stating that I need to delete the resource before creating the subnets. even though there is no change in the existing subnets

the part of the Subnet code in an array object, it should perform an incremental update instead it attempts to delete and recreate the subnets.

appreciate if this issue gets resolved as soon as possible,

[alex-frankel]

This change is not fully rolled out, but once it is you can deploy subnets as child resources and you won't run into this error anymore: https://techcommunity.microsoft.com/t5/azure-networking-blog/azure-virtual-network-now-supports-updates-without-subnet/ba-p/4067952

https://github.com/Azure/azure-quickstart-templates/issues/2786#issuecomment-2007870750

I don't believe it will fix the noise issue unfortunately. That is a separate problem.

[Sviatik]

Hello @alex-frankel,

Thank you for providing usefull links. But still I got problem with migrate subnets to separate resources. I can't even make bicep not to delete all my subnets. I tried to run with all options described in the topic above:

• subnets: []
• subnets: null
• Completely delete subnets from Microsoft.Network/virtualNetworks

• tried both api version 2023-09-01 and 2023-11-01

  Region is: westeurope Each of these tries had the same output

---

Bicep resource which was applied before:

resource vNet 'Microsoft.Network/virtualNetworks@2023-05-01' = {
  name: vNetName
  location: location
  properties: {
    addressSpace: {
      addressPrefixes: [
        vNetCidr
      ]
    }

    subnets: [

      {
        name: 'GatewaySubnet'
        properties: {
          addressPrefix: gatewaySubnetPrefix
        }
      }

      {
        name: bastionSubnetName
        properties: {
          addressPrefix: bastionSubnetPrefix
        }
      }
      {
        name: svcSubnetName
        properties: {
          addressPrefix: svcSubnetPrefix
        }
      }
      {
        name: webAppSubnetName
        properties: {
          addressPrefix: webAppSubnetPrefix
          delegations: [
            {
              name: 'delegation'
              properties: {
                serviceName: 'Microsoft.Web/serverFarms'
              }
            }
          ]
        }
      }
    ]
  } 

  tags: {
    env: envName
  }

}

Bicep resource resource I want to apply:

resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' = {
  name: vNetName
  location: location
  properties: {
    addressSpace: {
      addressPrefixes: [
        vNetCidr
      ]
    }
  }
  tags: {
    env: envName
  }
}

Appreciate of any help on this problem.

[alex-frankel]

@Sviatik - did you confirm with a real deployment that vnet is deleting subnets with all of the above settings? Or are you saying that ONLY the "what-if" output is not changing? I don't believe the what-if noise will go away regardless of the setting you choose.

Removing the subnets property completely is what you need to do to ensure subnets will not be removed. The other settings WILL delete subnets.

[Sviatik]

@alex-frankel Ahh, yes, it doesn't remove but in "plan" it was highlighted. After I removed unused and add needed/existing subnet as a separate resource it works well now.

Thank you.