This module is used to deploy subnet with network security group (NSG) and route table (RT) associated as workaround to the azurerm_subnet
resource so it doesn't conflict with the Azure Landing Zone policies Subnets should have a Network Security Group
and Subnets should have a User Defined Route
which are commonly used within large/medium sized enterprises in the Azure Landing Zone Reference Architecture.
Why not just use the terraform resource azurerm_subnet
to deploy the subnet within the virtual network deployed by the e.g Azure Landing Zone vending module ?
That is because it's known for having a bug that conflicts with these deny
policies due to how the resouce is designed, and how it's not able to associate the Network Security Group (NSG)
and the Route Table (RT)
to the subnet when the resouce is deployed, due to those restrictions in the resource it needs to use seperate resources for that action azurerm_subnet_network_security_group_association
and the azurerm_route_table_association
but unfortunately that conflicts with the Deny
policies and doesn't allow you to create the subnet.
The only way to deploy a subnet that doesn't conflict with these policies is to use the azurerm_virtual_network
resource to deploy the subnet and reference the network security group, and the route table in one API call.
However, that only resolves the part of the problem as the azurerm_virtual_network
resouce doesn't support complex configurations on the subnet like Subnet Delegation
, Service Endpoints
and Private link-/ endpoint network policies
In addition to prevent autonomous work teams/owners/contributors of their Landing Zone to create a subnet within their own code repository, and constant need to involve the Platform Team to create the subnets for the landing zones.
These github issues have been around since mid 2019, some of them have been resolved with workarounds, workaround similar to what is being used in this module, and some are still open.
Please have a look at the Known Issues section for more information, before posting an issue on this repository.
Name | Version |
---|---|
terraform | >= 1.3 |
azapi | >= 1.9, < 2.0 |
azurerm | >= 3.11, < 4.0 |
# Simple usage to showcase the functionality of the module
########################
# Pre-requisites Setup #
########################
resource "azurerm_resource_group" "this" {
name = "rg-test-01"
location = "westeurope"
}
resource "azurerm_virtual_network" "this" {
name = "vnet-alz-01"
address_space = ["10.30.0.0/24"]
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
}
# Pre-creating route table to showcase simple usage of the module
resource "azurerm_route_table" "this" {
name = "rt-alz-01"
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
}
# Pre-creating network security group to showcase simple usage of the module
resource "azurerm_network_security_group" "this" {
name = "nsg-alz-01"
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
}
########################
# Example Usage #
########################
module "subnet" {
source = "haflidif/alz-subnet/azurerm"
subnet_name = "snet-alz-01"
address_prefixes = ["10.30.0.0/27"]
virtual_network_resource_id = azurerm_virtual_network.this.id
location = azurerm_resource_group.this.location
create_network_security_group = false
create_route_table = false
route_table_id = azurerm_route_table.this.id
network_security_group_id = azurerm_network_security_group.this.id
}
:arrow_forward: Example: Creating Subnet, NSG, and Route Table in an Existing Resource Group
:information_source: Note:
Otherwise, see the full module test here: :arrow_forward: test/autotest
Name | Type |
---|---|
azapi_resource.subnet | resource |
azurerm_network_security_group.nsg | resource |
azurerm_resource_group.sub_resouces_rg | resource |
azurerm_route.default_route_to_nva | resource |
azurerm_route_table.route_table | resource |
Name | Description | Type | Default | Required |
---|---|---|---|---|
address_prefixes | (Required) The Address prefix to use for the subnet. | list(string) |
n/a | yes |
location | (Required) Specifies the Azure location where the resources should be created. Changing this forces a new resource to be created. | string |
n/a | yes |
subnet_name | (Required) Name of the subnet. Changing this forces a new resource to be created. | string |
n/a | yes |
virtual_network_resource_id | (Required) The ID of the virtual network where the subnet should be created. Changing this forces a new resource to be created. | string |
n/a | yes |
create_network_security_group | (Optional) Boolean flag which controls if network security group should be created. Defaults to true . Set to false and provide value for network_security_group_id to reference existing network security group. |
bool |
true |
no |
create_route_table | (Optional) Boolean flag which controls if route table should be created. Defaults to true . Set to false and either provide value for route_table_id to reference existing route table, or skip providing value for route_table_id to not use a route table at all. |
bool |
true |
no |
delegation_service_name | (Optional) Provide the service name for the subnet delegation configuration. | string |
"" |
no |
disable_bgp_route_propagation | (Optional) Boolean flag which controls propagation of routes learned by BGP on that route table. true means disable. Defaults to false , when used in combination with create_sub_network_resources and 'nva_ip_address' this should be set to true to prevent the default 0.0.0.0/0 route to be propagated in the route table via BGP. |
bool |
false |
no |
existing_resource_group_name | (Optional) The name of an existing resource group where the nsg and route table will be created. Changing this forces a new resource to be created. | string |
"" |
no |
network_security_group_id | (Optional) The ID of existing network security group to associate with the subnet. | string |
"" |
no |
network_security_group_name | (Optional) The name of the new network security group to associate with the subnet. | string |
"" |
no |
nva_ip_address | (Optional) The IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route to route 0.0.0.0/0 traffic to the network virtual appliance. | string |
"" |
no |
private_endpoint_network_policies_enabled | (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true will Enable the policy and setting this to false will Disable the policy. Defaults to true |
bool |
true |
no |
private_link_service_network_policies_enabled | (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true will Enable the policy and setting this to false will Disable the policy. Defaults to true |
bool |
true |
no |
route_table_id | (Optional) The ID of existing route table to associate with the subnet. | string |
"" |
no |
route_table_name | (Optional) The name of the new route table to associate with the subnet. | string |
"" |
no |
service_endpoint_names | (Optional) List of service endpoints to associate with the subnet. | list(string) |
[] |
no |
sub_resource_group_name | (Optional) The name of the resource group where the sub-resources will be created. Changing this forces a new resource to be created. | string |
"" |
no |
tags | (Optional) A mapping of tags to assign to the resource. | map(string) |
{} |
no |
use_existing_resource_group | (Optional) Boolean flag which controls if an existing resource group should be used. Defaults to false . |
bool |
false |
no |
Name | Description |
---|---|
subnet_id | Virtual Network Subnet resource id |
subnet_nsg_id | Network Security Group resource id |
subnet_route_table_id | Route Table resource id |
No modules.
The following arguments are supported:
subnet_name
- (Required) Name of the subnet. Changing this forces a new resource to be created.
address_prefixes
- (Required) The Address prefix to use for the subnet.
:information_source: NOTE:
Currently only a single address prefix can be set as the Multiple Subnet Address Prefixes Feature is not yet in public preview or general availability.
virtual_network_resource_id
- (Required) The ID of the virtual network where the subnet should be created. Changing this forces a new resource to be created.
location
- (Required) Specifies the Azure location where the resources should be created. Changing this forces a new resource to be created.
use_existing_resource_group
- (Optional) Boolean flag which controls if an existing resource group should be used for the NSG and Route Table. Defaults to false
.
existing_resource_group_name
- (Optional) The name of an existing resource group where the NSG and Route Table will be created. Changing this forces a new resource to be created.
create_network_security_group
- (Optional) Boolean flag which controls if network security group should be created. Defaults to true
. Set to false
and provide value for network_security_group_id
to reference existing network security group.
:information_source: NOTE:
If this is set tofalse
the following argument is requirednetwork_security_group_id
if it is notset
then the deployment will conflict with the Azure Policies and subnet can't be deployed in compliance with the policy.
network_security_group_id
- (Optional) The ID of existing network security group to associate with the subnet, make sure the create_sub_network_resources
is set to false
if you want to reference an existing network security group.
network_security_group_name
- (Optional) The name of the new network security group to associate with the subnet.
create_route_table
- (Optional) Boolean flag which controls if route table should be created. Defaults to true
. Set to false
and provide value for route_table_id
to reference existing route table.
:information_source: NOTE:
If this is set tofalse
the following argument is requiredroute_table_id
if it is notset
then the deployment will conflict with the Azure Policies and subnet can't be deployed in compliance with the policy.
route_table_id
- (Optional) The ID of existing route table to associate with the subnet.
route_table_name
- (Optional) The name of the new route table to associate with the subnet.
nva_ip_address
- (Optional) The IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route to route
:information_source: NOTE:
Specify the IP address of the network virtual appliance often a firewall, located in a hub virtual network, this is used to create user defined route in the new route table to route0.0.0.0/0
to the network virtual appliance.
sub_resource_group_name
- (Optional) The name of the resource group where the sub-resources will be created. Changing this forces a new resource to be created.
delegation_service_name
- (Optional) Provide the service name for the subnet delegation configuration.
:information_source: NOTE:
Delegating to service may not be available in all regions. Check if the service you are delegating to is available in your region using the [Azure CLI](https://learn.microsoft.com/cli/azure/network/vnet/subnet?view=azure-cli-latest#az-network-vnet-subnet-list-available-delegations()&wt.mc_id=SEC-MVP-5005265).
service_endpoint_names
- (Optional) List of service endpoints to associate with the subnet, multiple service endpoints are supported.
:information_source: NOTE:
In short Service Endpoints are used to secure Azure service resources to use the virtual network instead of the public internet, utilizing the azure backbone network. Multiple Service Endpoints can be defined on each subnet.
Generally available service endpoints for all regions are:
Microsoft.Storage
, Microsoft.Storage.Global
, Microsoft.Sql
, Microsoft.AzureCosmosDB
, Microsoft.KeyVault
, Microsoft.ServiceBus
, Microsoft.EventHub
, Microsoft.AzureActiveDirectory
, Microsoft.Web
, Microsoft.CognitiveServices
Public Preview:
Microsoft.ContainerRegistry
For the most up-to-date notification and list of available service endpoints in your region check the Azure virtual network service endpoints documentation.
private_link_service_network_policies_enabled
- (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true
will Enable the policy and setting this to false
will Disable the policy. Defaults to true
:information_source: NOTE:
Private Link Service Network Policies are enabled by default to ensure that traffic from Private Link services go through the Network Security Group and uses the User Defined Routes in the route table associated with the subnet.
If this is set tofalse
the traffic for all private link services will bypass the Network Security Group and the User Defined Routes in the route table associated with the subnet.
private_endpoint_network_policies_enabled
- (Optional) Enable or Disable network policies for the private endpoint on the subnet. Setting this to true
will Enable the policy and setting this to false
will Disable the policy. Defaults to true
:information_source: NOTE:
Private Endpoint Network Policies are enabled by default to ensure that traffic from the private endpoint go through the Network Security Group and uses the User Defined Routes in the route table associated with the subnet.
If this is set tofalse
the traffic for all private endpoints in the subnet will bypass the Network Security Group and the User Defined Routes in the route table associated with the subnet.
disable_bgp_route_propagation
- (Optional) Boolean flag which controls propagation of routes learned by BGP on that route table. true
means disable. Defaults to false
, when used in combination with create_sub_network_resources
and nva_ip_address
this should be set to true
to override the default route 0.0.0.0/0
and to prevent routes learned by BGP (Route Propagation) to bypass the network virtual appliance.
tags
- (Optional) A mapping of tags to assign to the resource.
Originally created by Haflidi Fridthjofsson
The Azure API might throw an Response 409: 409 Conflict
, Error Code: AnotherOperationInProgress
when creating the subnet, this is due to dely in the Azure API when the virtual network
resource is being updated and another create/update/delete
is ran at the same time, in parallel or before the Azure API catches up with the previous operation.
| --------------------------------------------------------------------------------
│ RESPONSE 409: 409 Conflict
│ ERROR CODE: AnotherOperationInProgress
│ --------------------------------------------------------------------------------
│ {
│ "error": {
│ "code": "AnotherOperationInProgress",
│ "message": "Another operation on this or dependent resource is in progress. To retrieve status of the operation use uri: https://management.azure.com/subscriptions/<GUID_REMOVED>/providers/Microsoft.Network/locations/westeurope/operations/<GUID_REMOVED>?api-version=2023-04-01.",
│ "details": []
│ }
│ }
│ --------------------------------------------------------------------------------
This happens particularly when calling the module multiple times in the same terraform run, the workaround is to use explicit depends_on
on the module resource to ensure that the module is ran sequentially e.g.
module "subnet1" {
source = "haflidif/alz-subnet/azurerm"
...ommitted for brevity
}
module "subnet2" {
source = "haflidif/alz-subnet/azurerm"
...ommitted for brevity
depends_on = [ module.subnet1 ]
}
See the following testing mechanism being used to test the module with multiple subnets in the same terraform run:
:arrow_forward: test/autotest
Please submit a issue on this repository if you find a better workaround or solution to this issue.
:information_source: NOTE:
It's a known issue not specifically related to this module, but to theazurerm
provider and theAzure API
and is being tracked in the following github issue: :arrow_forward: Subnets on same vnet fail due to parrallel setup #3780