Azure / terraform-azurerm-caf-enterprise-scale

Azure landing zones Terraform module
https://aka.ms/alz/tf
MIT License
857 stars 567 forks source link

DDoS protection plan will not be removed #582

Closed skiddder closed 1 year ago

skiddder commented 1 year ago

Community Note

Versions

terraform: 1.3.6

azure provider: 3.37.0

module: 3.1.2

Description

DDoS protection plan will not be removed when it was deployed before

When turning the switch in ddos_protection_plan.enabled from true to false, I receive the error message in terraform:

Error: deleting Ddos Protection Plan: (Name "es-ddos-northeurope" / Resource Group "es-ddos"): network.DdosProtectionPlansClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="InUseDdosProtectionPlanCannotBeDeleted" Message="DDoS Protection Plan /subscriptions/aacc8343-56ad-4b3f-9986-de980868f677/resourceGroups/es-ddos/providers/Microsoft.Network/ddosProtectionPlans/es-ddos-northeurope cannot be deleted because it is in use with virtual networks /subscriptions/aacc8343-56ad-4b3f-9986-de980868f677/resourceGroups/es-connectivity-westeurope/providers/Microsoft.Network/virtualNetworks/es-hub-westeurope. Please disable DDoS standard on the virtual network as explained in the documentation link : [https://docs.microsoft.com/en-us/azure/virtual-network/manage-ddos-protection#disable-ddos-for-a-virtual-network."](https://docs.microsoft.com/en-us/azure/virtual-network/manage-ddos-protection#disable-ddos-for-a-virtual-network.%22) Details=[]

I checked via Azure portal whether there are any protected resources in ddos protection plan. And I found that there is the hub vnet and the vnet gateway still under protection:

image

image

My expectation was that protections that were automatically created by the module will be removed when removing the ddos protection plan.

I did not create any other protectable resources, but I think it might be worth validating that in case you have created a firewall via the module, that this protection is also removed automatically.

Steps to Reproduce

While doing a greenfield deployment in my lab, I used this settings.connectivity.tf file:


locals {
  configure_connectivity_resources = {
    settings = {
      hub_networks = [
        {
          enabled = true
          config = {
            address_space                = ["10.101.0.0/16", ]
            location                     = "westeurope"
            link_to_ddos_protection_plan = true
            dns_servers                  = []
            bgp_community                = ""
            subnets                      = []
            virtual_network_gateway = {
              enabled = true
              config = {
                address_prefix           = "10.101.1.0/24"
                gateway_sku_expressroute = ""
                gateway_sku_vpn          = "VpnGw1" 
                advanced_vpn_settings = {
                  enable_bgp                       = null
                  active_active                    = null
                  private_ip_address_allocation    = ""
                  default_local_network_gateway_id = ""
                  vpn_client_configuration         = []
                  bgp_settings                     = []
                  custom_route                     = []
                }
              }
            }
            azure_firewall = {
              enabled = false
              config = {
                address_prefix                = ""
                enable_dns_proxy              = true
                dns_servers                   = []
                sku_tier                      = ""
                base_policy_id                = ""
                private_ip_ranges             = []
                threat_intelligence_mode      = ""
                threat_intelligence_allowlist = []
                availability_zones = {
                  zone_1 = true
                  zone_2 = true
                  zone_3 = true
                }
              }
            }
            spoke_virtual_network_resource_ids      = []
            enable_outbound_virtual_network_peering = true
            enable_hub_network_mesh_peering         = false
          }
        },
      ]
      vwan_hub_networks = []
      ddos_protection_plan = {
        enabled = true
        config = {
          location = "northeurope"
        }
      }
      dns = {
        enabled = true
        config = {
          location = null
          enable_private_link_by_service = {
            azure_api_management                 = true
            azure_app_configuration_stores       = true
            azure_arc                            = true
            azure_automation_dscandhybridworker  = true
            azure_automation_webhook             = true
            azure_backup                         = true
            azure_batch_account                  = true
            azure_bot_service_bot                = true
            azure_bot_service_token              = true
            azure_cache_for_redis                = true
            azure_cache_for_redis_enterprise     = true
            azure_container_registry             = true
            azure_cosmos_db_cassandra            = true
            azure_cosmos_db_gremlin              = true
            azure_cosmos_db_mongodb              = true
            azure_cosmos_db_sql                  = true
            azure_cosmos_db_table                = true
            azure_data_explorer                  = true
            azure_data_factory                   = true
            azure_data_factory_portal            = true
            azure_data_health_data_services      = true
            azure_data_lake_file_system_gen2     = true
            azure_database_for_mariadb_server    = true
            azure_database_for_mysql_server      = true
            azure_database_for_postgresql_server = true
            azure_digital_twins                  = true
            azure_event_grid_domain              = true
            azure_event_grid_topic               = true
            azure_event_hubs_namespace           = true
            azure_file_sync                      = true
            azure_hdinsights                     = true
            azure_iot_dps                        = true
            azure_iot_hub                        = true
            azure_key_vault                      = true
            azure_key_vault_managed_hsm          = true
            azure_kubernetes_service_management  = true
            azure_machine_learning_workspace     = true
            azure_managed_disks                  = true
            azure_media_services                 = true
            azure_migrate                        = true
            azure_monitor                        = true
            azure_purview_account                = true
            azure_purview_studio                 = true
            azure_relay_namespace                = true
            azure_search_service                 = true
            azure_service_bus_namespace          = true
            azure_site_recovery                  = true
            azure_sql_database_sqlserver         = true
            azure_synapse_analytics_dev          = true
            azure_synapse_analytics_sql          = true
            azure_synapse_studio                 = true
            azure_web_apps_sites                 = true
            azure_web_apps_static_sites          = true
            cognitive_services_account           = true
            microsoft_power_bi                   = true
            signalr                              = true
            signalr_webpubsub                    = true
            storage_account_blob                 = true
            storage_account_file                 = true
            storage_account_queue                = true
            storage_account_table                = true
            storage_account_web                  = true
          }
          private_link_locations = [
            "westeurope",
          ]
          public_dns_zones                                       = []
          private_dns_zones                                      = []
          enable_private_dns_zone_virtual_network_link_on_hubs   = true
          enable_private_dns_zone_virtual_network_link_on_spokes = true
          virtual_network_resource_ids_to_link                   = []
        }
      }
    }
    location = var.connectivity_resources_location
    tags     = var.connectivity_resources_tags
    advanced = null
  }
}

I then saw that ddos protection plan will probably bring my budget to the limit, so I decided to switch it off in my lab by changing the ddos_protection_plan.enabled property in settings.connectivity.tf to false:

      vwan_hub_networks = []
      ddos_protection_plan = {
        enabled = false
        config = {
          location = "northeurope"
        }
      }

When I ran into the error above, I checked whether there are switches to disable the protection on hub vnet and vpn gateway. I found that there is a switch for hub vnet which I also set to "false". For vpn gateway I couldn't find such a switch. But I still run into the same error as terraform is first trying to delete the ddos_protection before changing the protection setting on the hub vnet.

Screenshots

Additional context

I am running terraform via an Azure DevOps pipeline.

jtracey93 commented 1 year ago

Hey @skiddder,

Thanks for the issue. I think if you also set the link_to_ddos_protection_plan to false as part of the hub networks block this will allow you to disable it.

As the DDoS protection of the VPN GW is actually coming from the fact the Hub VNet has DDoS link enabled from the above variable.

So changing this to false as well as the DDoS plan disablement as you have done should do the trick here.

skiddder commented 1 year ago

Hi @jtracey93 ,

I think if you also set the link_to_ddos_protection_plan to false as part of the hub networks block this will allow you to disable it.

I already tried this.

After running the pipeline several times with different combinations of true/false for link_to_ddos_protection_plan and ddos_protection_plan.enabled it finally went through without error. Having to run it several times seems to be an indicator that terraform does not recognize the dependancy between those resources.

So as a best practice, one should deactivate the protection first and in a 2nd step remove the ddos_protection plan itself.