Create VNET without destroying all subnets

[gregjhogan]

I want to create subnets dynamically (separately from my VNET). I see there is a way to create a subnet inside an existing VNET, indicating that a subnet is a child resource of a VNET.

https://github.com/Azure/azure-quickstart-templates/blob/master/101-subnet-add-vnet-existing/azuredeploy.json

However, I can't find a way to re-deploy the VNET template without blowing away all subnets, because if I leave off the subnets property it deletes all the VNETs. Is there a way to deploy a VNET without including (and destroying) the subnets?

Here is an example template deploying a VNET (which destroys all subnets)

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "vnetAddressPrefixes": {
            "type": "array"
        }
    },
    "variables": {
        "apiVersion": "2015-06-15",
        "namePrefix": "[concat(replace(subscription().displayName, ' ', '_'), '_')]",
        "vnetName": "[concat(variables('namePrefix'), 'VNET')]"
    },
    "resources": [
        {
            "apiVersion": "[variables('apiVersion')]",
            "type": "Microsoft.Network/virtualNetworks",
            "name": "[variables('vnetName')]",
            "location": "[resourceGroup().location]",
            "dependsOn": [],
            "properties": {
                "addressSpace": {
                    "addressPrefixes": "[parameters('vnetAddressPrefixes')]"
                }
            }
        }
    ]
}

[MCKLMT]

I guess you should use the incremental mode: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy#incremental-and-complete-deployments But in the case of the VNet, I think you should always define the Subnets in the template with the VNet.

[gregjhogan]

@MCKLMT I want to have a single shared VNET (template) that application teams cannot change, and I want to give each application their own subnet. Ideally my templates would be separated by who can change them (VNET template can be changed by one group, subnet templates each by different groups). The VNET template also has gateways, public IP addresses, etc... This causes me to have to manage the VNET without a template (because it will always blow away all the subnets when deployed, even in incremental mode).

I understand this isn't currently supported, but why? It seems very inconsistent considering subnets are technically resources that stand on their own.

[MCKLMT]

A subnet takes part of a VNet so it's legitimate to found them in the same template than the VNet. I believe you should think about another way to achieve your goal.

[gregjhogan]

@MCKLMT can you think of any other set of resources with a child-parent relationship that work like that? There is no way to re-deploy an app service plan and destroy the web sites it contains (in incremental mode). There is no way to deploy an Azure SQL server and destroy the databases it contains (in incremental mode). Why are VNETs and subnets different?

There is currently ARM template support for creating a subnet as a stand-alone resource (VNET already exists), so why does it seem acceptable that re-deploying the VNET destroys all subnets if the subnets are omitted?

[MCKLMT]

Because all the subnets should be defined in the template. You can't add subnets, one by one.

[gregjhogan]

@MCKLMT you are wrong, here is the example: https://github.com/Azure/azure-quickstart-templates/blob/master/101-subnet-add-vnet-existing/azuredeploy.json

[MCKLMT]

You're right. But you can't define the subnets and the VNet in separate files in a deployment.

[nik-shornikov]

@MCKLMT 👎 -- we can see that, but why the heck not? It doesn't matter if they're in the same file. If they're separate resources, it fails.

Why is the child resource even exposed if the properties are going to remove it on the next run (incremental or not)?

Pick your bug (and your container hierarchy). Either exposing subnets as standalone resources is a bug, or unspecified properties equating to null properties.

Here's a lovely example: 301-subnet-driven-deployment. For all the BS workarounds it exemplifies for not having the deployment objects wipe the subnets created by the previous copy (a parametrized number of times), there's no way to reference a potentially existing resource, is there? So, there's no way to rerun it, for all its overengineered glory.

Trivial constructive repro here: https://github.com/Azure/azure-quickstart-templates/issues/2897

[gregjhogan]

Subnets sure seem like stand-alone resources to me because (a) you can assign RBAC to them and (b) you can create them separately from the VNET. If that is true, to be consistent (and sane) deploying a parent resource by itself should not destroy child resources (just like Azure SQL server/database). I think a mistake was made when subnets were added as a property of the VNET (it should exclusively be a child resource).

[mchute84]

Did you ever get a resolution to this? I have come across the same issue and been referenced this link. I too can create the vNet and subnets as seperate templates.

Annoyingly I have the vNet and some Core infrastructure setup as part of a "Infrastructure Architecture" project. I then use a separate "Deploy a Subnet" step which creates a subnet as part of a Solution Project. However its frustrating that it deletes all the subnets (and associated settings, VPN connections, express route, NSG's etc... ) each time causing no end of issues.

Is there a way to link steps between projects at all ?

[CtrlDot]

This is ABSOLUTELY an issue. Can Microsoft please address?

I am running my templates in incremental mode which claims that resources not defined in the template will not be deleted. However, subnets missing in the original template WILL be deleted.

Can someone from MS please comment?

Thanks,

[singhkays]

@CtrlDot I will pass this on to the networking team but I would also suggest creating a support request for this instead so you can reach the networking team directly on the expected behavior. This seems like a Networking resource provider issue rather than a template issue.

[gregjhogan]

https://feedback.azure.com/forums/217313-networking/suggestions/18758545-support-vnet-re-deployment-without-destroying-subn

[anavinahar]

I want to highlight that our documentation (https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy) states the following: 'In incremental mode, Resource Manager leaves unchanged resources that exist in the resource group but are not specified in the template.' I understand that since subnets are a property of virtual networks, of course virtual network has to be specified when trying to update the subnet, Thus, the virtual network will not remain unchanged in incremental mode. A way work with this currently, if you would like to use templates and not PowerShell, CLI or Portal, would be to do a GET so you have most updated information about your virtual network without running the risk of resource deletion. Thank you for the feedback, we will look into it.

[mthoger]

Is there any news on this subject ? It is a major issue.. I have Loops, generating subnetnames, I have no way of knowing the actual name of the Subnet, nsg beforehand. This was smart and functional, until I had to rerun my template.. and is a major design flaw.

Same goes for NSGs, as this has to be attached to the subnet ( a property on the subnet, not the NSG ). Så having a preset infrastructure (hub and spoke ), adding subnets, deletes the existing subnet. and there is no way to add a NSG to the subnet, as the NSG is deployed with the Application resources not the VNet ( hub and spoke setup infrastructure ).

We will need a property on the NSG to attach the Subnet, and will need Subnets to be fully acknowledable atm template objects, with a property to attach it to the Vnet.

you Networks guys, turned this the wrong way around.

[ambimanus]

+1

[rmodayil]

Any update on this issue. I have the same problem when using 2 seperate nested templates for creating Vnet and Creating subnets, every time you re-deploy it tries to delete the subnets

[tdrimmelen]

+1 Having this issue when using a "virtualNetworks" resource without any subnet and a separate "virtualNetwork/subnets" resources to define all subnets (using a "copy" property to create all the subnets).

[kensykora]

+1 this issue.

One of the comments above by @MCKLMT

But you can't define the subnets and the VNet in separate files in a deployment.

This is not true, you can absolutely define the subnets in separate deployment files. Our scenario is that for some instances of our application we need a virtual network gateway for us to connect into (our production subscription), and in some scenarios we do not (our development & UAT subscriptions) -- We would like to achieve this by embedding a parameter in our ARM template that includes (or excludes) a virtual network gateway (and subnet!) based on this flag.

When I try to re-deploy, even in incremental mode, the ARM deployment attempts to destroy the gateway subnet. Deploying from a clean slate to a new resource group works just fine though.

My workaround is to include the gateway in the virtual network, regardless of if it is needed. While this is harmless, still feels frustrating that this isn't supported.

-- Here's an example :

azuredeploy.json

{
"parameters": {
    "includeVpnGateway": {
            "type": "string",
            "defaultValue": "no",
            "allowedValues": [
                "yes",
                "no"
            ]
        }
},
...
resources: [{
            "name": "[variables('virtualNetworkName')]",
            "apiVersion": "2017-03-01",
            "type": "Microsoft.Network/virtualNetworks",
            "location": "[resourceGroup().location]",
            "tags": {
                "displayName": "[variables('virtualNetworkName')]"
            },
            "properties": {
                "addressSpace": {
                    "addressPrefixes": [
                        "[concat(parameters('vnetAddressSpacePrefix'),variables('addressSpaceSuffix'))]"
                    ]
                },
                "subnets": [
                    {
                        "name": "PrivateSubnet",
                        "properties": {
                            "addressPrefix": "[concat(parameters('vnetAddressSpacePrefix'),variables('privateSubnetAddressSpaceSuffix'))]"
                        }
                    },
                    {
                        "name": "DMZSubnet",
                        "properties": {
                            "addressPrefix": "[concat(parameters('vnetAddressSpacePrefix'),variables('dmzSubnetAddressSpaceSuffix'))]"
                        }
                    }
                ]
            }
        },
        {
            "apiVersion": "2015-01-01",
            "name": "VpnGateway",
            "type": "Microsoft.Resources/deployments",
            "comments": "Adds VPN gateway if the `includeVpnGateway` parameter is set to 'yes'",
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[variables('gatewayTemplateLink')]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "resourcePrefix": {
                        "value": "[parameters('resourcePrefix')]"
                    },
                    "vnetAddressSpacePrefix": {
                        "value": "[parameters('vnetAddressSpacePrefix')]"
                    },
                   "virtualNetworkName": {
                        "value": "[variables('virtualNetworkName')]"
                    }
                }
            },
            "dependsOn": [
                "[variables('virtualNetworkName')]"
            ]
        }
...
]
}
...

And a separate file for deploying a virtual network gateway -

_vpn-gateway-yes.json

{
    ... 
resources: [{
            "name": "[concat(parameters('virtualNetworkName'),'/', variables('gatewaySubnetName'))]",
            "type": "Microsoft.Network/virtualNetworks/subnets",
            "apiVersion": "2017-03-01",
            "properties": {
                "addressPrefix": "[concat(parameters('vnetAddressSpacePrefix'), variables('gatewaySubnetAddressSpaceSuffix'))]"
            }
        },
        {
            "apiVersion": "2015-06-15",
            "type": "Microsoft.Network/publicIPAddresses",
            "name": "[variables('gatewayPublicIpName')]",
            "location": "[resourceGroup().location]",
            "tags": {
                "displayName": "[variables('gatewayPublicIpName')]"
            },
            "properties": {
                "publicIPAllocationMethod": "Dynamic",
                "dnsSettings": {
                    "domainNameLabel": "[toLower(variables('gatewayPublicIpName'))]"
                }
            }
        },
        {
            "apiVersion": "2015-06-15",
            "name": "[variables('gatewayName')]",
            "type": "Microsoft.Network/virtualNetworkGateways",
            "location": "[resourceGroup().location]",
            "dependsOn": [
                "[concat('Microsoft.Network/publicIPAddresses/', variables('gatewayPublicIpName'))]",
                "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworkName'), variables('gatewaySubnetName'))]"
            ],
            "properties": {
                "gatewayType": "Vpn",
                "ipConfigurations": [{
                    "name": "default",
                    "properties": {
                        "privateIPAllocationMethod": "Dynamic",
                        "subnet": {
                            "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworkName'), variables('gatewaySubnetName'))]"
                        },
                        "publicIpAddress": {
                            "id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('gatewayPublicIpName'))]"
                        }
                    }
                }],
                "enableBgp": false,
                "vpnType": "RouteBased",
                "sku": {
                    "name": "VpnGw1",
                    "tier": "VpnGw1"
                }
            }
        }
    ]
    ...
}

[jabbera]

We have this issue too. We would love to manage each subnet separately.

[jabbera]

@mthoger you can use the copy attribute on properties now to work around this.

 {
      "apiVersion": "2015-06-15",
      "type": "Microsoft.Network/virtualNetworks",
      "name": "[variables('vnetName')]",
      "location": "[resourceGroup().location]",
      "dependsOn": [
        "routeTableCopy",
      ],
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "[parameters('vnetAddressPrefix')]"
          ]
        },        
        },
        "copy": [
          {
            "name": "subnets",
            "count": "[length(variables('subnets'))]",
            "input": {
              "name": "[variables('subnets')[copyIndex('subnets')].name]",
              "properties": {
                "addressPrefix": "[variables('subnets')[copyIndex('subnets')].prefix]",
                "routeTable": {
                  "id": "[resourceId('Microsoft.Network/routeTables', concat(variables('vnetPrefix'), variables('subnets')[copyIndex('subnets')].name, '-rt'))]"
                }
              }
            }
          },
 {
      "type": "Microsoft.Network/routeTables",
      "apiVersion": "2015-06-15",
      "location": "[resourceGroup().location]",
      "comments": "We do sub -1 to exclude the gateway subnet. That needs special UDR\\RT",
      "copy": {
        "name": "routeTableCopy",
        "count": "[sub(length(variables('subnets')), 1)]"
      },
      "name": "[concat(variables('vnetPrefix'), variables('subnets')[copyIndex()].name, '-rt')]",
      "properties": {
        "routes": [
          {
            "name": "DefaultRoute",
            "properties": {
              "addressPrefix": "0.0.0.0/0",
              "nextHopType": "VirtualNetworkGateway"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/routeTables",
      "apiVersion": "2015-06-15",
      "location": "[resourceGroup().location]",
      "name": "[variables('gatewaySubnetRtName')]",
      "properties": {
        "routes": []
      }
    }

[kamoljan]

+1

[allanmoller]

Hey found a way to achieve this, when deploying VNET i do a condition ( "condition": "[equals(parameters('rebuildVNET'),'Yes')]",) i then set it to default "no" in parameters, and have all subnet templates deployed as nested templates. This allows me to edit NSGs and add subnets in a running VNET with a connected VM, if needed i can rebuild using the condition but this would require subnets to be empty.

Attached files as ZIP VNETman.zip

[MCKLMT]

Please create a pull request creating a new template based on you discoveries.

[jonathaneckman]

This is a big limitation for us. After our VNET is used, we are no longer able to deploy changes to it because it is attempting to delete our subnets which are created in a separate deployment.

[jamesbjackson]

I thinking we may just have to give up on ARM Templates at this rate and look at Terraform. Why can't ARM Templates work like CloudFormation Stacks and only update the properties defined.

[rdtechie]

To expand further on this issue, subnets in itself can also have some additional properties set like associated NSG, Routetable and Service Endpoints.

Let's take this example: I have a vnet, in this vnet I have 1 subnet with a routetable configured. Then I decide that I want to add an additional subnet. I create my ARM template. In this template I'm specifying the vnet, the existing subnet and the new subnet.

All good.

But, I forgot to specify in the ARM template that the existing subnet already has a routetable configured. I deploy the template, and boom, gone is my configured routetable on the existing subnet. This is really an issue, because the increment mode in my opinion should only update those things that you specify, and not delete/remove them.

If I really wanted to un-configure those type of properties, then I should pass some kind of $null value.

The main point here, once set, don't modify unless specified.

PS: I see the same behavior when I'm using PowerShell, with Set-AzureRmVirtualNetworkSubnetConfig. It just removes properties set like routetable and service endpoints.

[kensykora]

@rdtechie I disagree about your $null idea, that’d be wildly inconsistent with other Azure services and ARM templates. I think the ideal solution here is to treat subjects like a sub resource and not a property of a vnet resource

On Tue, Aug 21, 2018 at 3:34 PM Richard Diphoorn notifications@github.com wrote:

To expand further on this issue, subnets in itself can also have some additional properties set like associated NSG, Routetable and Service Endpoints.

Let's take this example: I have a vnet, in this vnet I have 1 subnet with a routetable configured. Then I decide that I want to add an additional subnet. I create my ARM template. In this template I'm specifying the vnet, the existing subnet and the new subnet.

All good.

But, I forgot to specify in the ARM template that the existing subnet already has a routetable configured. I deploy the template, and boom, gone is my configured routetable on the existing subnet. This is really an issue, because the increment mode in my opinion should only update those things that you specify, and not delete/remove them.

If I really wanted to un-configure those type of properties, then I should pass some kind of $null value.

The main point here, once set, don't modify unless specified.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Azure/azure-quickstart-templates/issues/2786#issuecomment-414812592, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD89N2To_YtxBkmZHqkoFKi1--Y14VYks5uTG7JgaJpZM4K4sjJ .

[rdtechie]

Agree @kensykora , but that wasn't my main point. The issue that should be prevented is that sub-resources should not be modified unless specified.

[jamesbjackson]

Well this is a bug from reading how Incremental is supposed to work. https://blogs.msdn.microsoft.com/mvpawardprogram/2018/05/01/azure-resource-manager/

"Incremental is the default mode. It only deploys the new resources in the ARM template. No resources are removed. So, if you have renamed an SQL Server database, the database you created earlier will still exist after applying the ARM template with the new database."

So this issue I seen also applies to anything that has a child parent relationship, DNS, Traffic Manager, VNET, Subnets, list goes on.

[mifurm]

I agree this is a bug but the workaround, which is easy to implement and working like a charm with "condition": "[equals(parameters('rebuildVNET'),'Yes')]" solves the issue. So, it is a bug but workaround exists :)

[nik-shornikov]

Since the original issue, we got parallelization controls, which helps in this general area. Thank you for that. This workaround is the very-very-near equivalent of just not having the VNET in the template, and presence/absence of resource config might be less surprising to a maintainer. It's not proper convergence in incremental mode and in complete mode.. how would it work exactly? A better workaround might end in a .tf, but man, two years!

[dirkslab]

Any update or news on this issue? It's now 2019 and this thread started in 2016. Very frustrating trying to automate deployments, to find that one resource have different rules to other resources. The bug should be fixed or subnet resource should be removed as a standalone template option. At a minimum a great big warning on the documentation, or have I missed this.

[mthoger]

This goes for peerings, nsg, udr and the list goes on and on and on ...

[patpicos]

Interestingly, Azure KeyVault can be redeployed and it wont delete secrets in it, even though they can be sub-resources or independent resources. It would be nice if VNET & Subnets had that same behavior

[mjcarrabine]

This behavior is inconsistent, and a big headache.

I created a template for my virtualNetwork (which includes addressPrefixes). Then I created a template for my networkSecurityGroup. Then I created a template for my subnet.

Then I needed to add a new addressPrefix to my virtualNetwork, so I added it to my virtualNetwork template. Only now, like everyone else above, it deletes my other subnets.

I agree that either:

1. re-deploying a virtualNetwork template should not delete all the existing subnets
2. OR subnets should not be deployable in their own template

I would prefer 1. because then it would behave like Key Vaults and Secrets.

[anavinahar]

Hi folks! I'm the VNet PM, I totally see how this looks like the team hasn't made any progress on this issue since it was first opened. Sorry for the trouble.

The issue here is that VNet APIs don't support PATCH. We are working to see when we can add this functionality. Even if we did though, the other problem with PATCH today is there isn’t a way for a template author to say to do a patch as templates always do PUTS. That being said we thought it would be best if we had a clear doc/guidance on VNets/subnets approaches for applying typical changes both through the API and through templates where we can show the right approach for using linked templates for subnets and the right conditionals in the main template to power their scenarios. Without this explicit guidance it is clearly hard to figure this out and you shouldn't have to figure it out yourselves.

In the mean time, please use workarounds mentioned in the thread and we will keep you posted.

Thank you!

[gregjhogan]

@anavinahar don't implement patch, remove the subnets property from Microsoft.Network/virtualNetworks and require using the Microsoft.Network/virtualNetworks/subnets type for subnets (using the resources property if nesting). Make it consistent with all other resource types with parent/child relationship (azure databases, web apps, etc...)

If this is too drastic of a change, implement the above suggestion except if the subnets property is missing don't touch (or destroy) the subnets, and if the subnets property is present assume it is the complete list of subnets (preserving current functionality)

[kensykora]

I agree with @gregjhogan -- make subnet it's own sub resource under a vnet. Understanding this might require a breaking API change but it would simplify a lot around subnet management.

[farroar]

Adding my .02 to this as well. I'm trying to add VNet peering after the fact. When those templates are applied, the subnets within the VNets I'm trying to peer are removed. I'm still new to ARM templates. I'm actually using Python and Jinja2 to build the templates off of a .csv and have one for the general environment and a separate one for just peering. I am looking at the workaround but having trouble getting it to fit within my process. Would love if I didn't have to change everything to make this work.

[chelnak]

+1 A subnet should be it's own resource.

[ljtill]

Recently we've also encountered this issue and had been using the condition patch to get around it. However after researching this issue further, I discovered the Stack Overflow link below and whereby the recommendation to place subnets configuration within the properties section of your virtual network rather a child resource / separate resource and this in turn resolved it for us.

https://stackoverflow.com/questions/52626721/subnet-azurefirewallsubnet-is-in-use-and-cannot-be-deleted

[JBrLloyd]

I'll add to @allanmoller's workaround to still allow the deployment of the VNet if it doesn't exist (as use the boolean type), bypassing the parameter so you don't have to change the parameter before a new vnet deployment each time: https://gist.github.com/JBrLloyd/557eb203e58b99692b4532e57469e35e

[Tadimsky]

@JBrLloyd it looks like you Gist is 404ing for me, would you mind double checking as I'd love to see what you did.

[coote1978]

The link looks broken could you provide the json template files which you have produced?

[loekd]

I believe you can work around this problem by specifying the subnets in both the VNet property and as separate resources, when using separate subnet resources. This way, I was able to add a subnet with delegation to an existing VNet, without recreating all old subnets.

[bytemech]

This is bloody ridiculous, 3 years... no fix.

[frankzo]

Can we get an update on this one?

[Tadimsky]

There was no interest from the team in fixing this when I spoke to them about it. So I wouldn't hold my breath :(

[bowlerma]

Just fell foul of this as well. :( Whilst you can declare the subnets as properties of the VNet, as long as you don't mind everything being declared within the same template, it's not as flexible particularly if you are using copy loops to create a number of subnets as the subnet object is quite a complex object.