Upgrade Azure Firewall to Premium in FORGE

[AErmie]

As part of our network security requirements, we will be exploring the use of Azure Firewall Premium features.

This requires updating the existing CAF Terraform module settings (specifically the settings.connectivity.tf file), and changing the Azure Firewall SKU to "Premium".

            azure_firewall = {
              enabled = true
              config = {
                enable_dns_proxy = true
                dns_servers = [
                  "10.41.12.4" # Prvate IP address of Private DNS Resolver inbound endpoint
                ]
                sku_tier                      = "Premium"
                base_policy_id                = ""
                private_ip_ranges             = []
                threat_intelligence_mode      = ""
                threat_intelligence_allowlist = {}
                availability_zones = {
                  zone_1 = true
                  zone_2 = true
                  zone_3 = true
                }
              }
            }

[AErmie]

Upgrade Observations

Terraform Plan

After changing the sku_tier property to Premium, terraform plan it shows that the Azure Firewall resource can be changed "in-place" (aka non-destructive), but, the Firewall Policy is a destructive upgrade. Notice the most important point, that the Rule Collection Groups will be removed (because we can't control those through the CAF deployment).

Terraform will perform the following actions:

  # module.connectivity.module.alz.azurerm_firewall.virtual_wan["/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral"] will be updated in-place
  ~ resource "azurerm_firewall" "virtual_wan" {
        id                  = "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral"
        name                = "bcgov-managed-lz-forge-fw-hub-canadacentral"
      ~ sku_tier            = "Standard" -> "Premium"
        tags                = {
            "deployedBy" = "azure-lz-core-forge"
        }
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.connectivity.module.alz.azurerm_firewall_policy.virtual_wan["/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy"] must be replaced
-/+ resource "azurerm_firewall_policy" "virtual_wan" {
      - auto_learn_private_ranges_enabled = false -> null
      ~ child_policies                    = [] -> (known after apply)
      ~ firewalls                         = [
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral",
        ] -> (known after apply)
      ~ id                                = "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy" -> (known after apply)
        name                              = "bcgov-managed-lz-forge-fw-hub-canadacentral-policy"
      - private_ip_ranges                 = [] -> null
      ~ rule_collection_groups            = [
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/WindowsAdminCenter_RuleCollectionGroup",
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/e833c2_RuleCollectionGroup",
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/Core_RuleCollectionGroup",
        ] -> (known after apply)
      ~ sku                               = "Standard" -> "Premium" # forces replacement
        tags                              = {
            "deployedBy" = "azure-lz-core-forge"
        }
        # (3 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Terraform Apply

When applying the above listed changes, the following error was thrown:

│ Error: deleting Firewall Policy (Subscription: "09bd024b-fbda-417d-b8db-694680c2b44e"
│ Resource Group Name: "bcgov-managed-lz-forge-connectivity"
│ Firewall Policy Name: "bcgov-managed-lz-forge-fw-hub-canadacentral-policy"): performing Delete: unexpected status 400 (400 Bad Request) with error: FirewallPolicyHasAzureFirewallReferences: Firewall Policy '/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy' can not be deleted since there are Azure Firewalls using this policy.

According to the following GitHub Issue (Unable to change the firewall SKU from Standard to Premium), it states

It seems that we you might need to manually unlink the policy from the firewall to get around this. We will look into fixing this in a future release.

[!NOTE] It is unknown at this time if this issue has been fixed in the CAF Terraform module, though it appears it has not been. Additionally, the original GitHub Issue was closed due to lack of activity. We may want to open a new GitHub Issue to re-surface this issue.

How-To Unlink Azure Firewall Policy

To "unlink" (aka disassociate) an Azure Firewall Policy from an existing Azure Firewall, in the portal navigate to Firewall Manager > Firewall Policy > Manage Associations > Remove Hub Association.

[!WARN] The wording on the UI can be misleading and cause concern. For example, "Remove Hub Association" may sound like you're removing the Firewall (not the policy) from the vHub. Also, the UI says "Remove" whereas the warning info says "Deleting policy associated..." Further, the bottom of the dialog also shows "Total VNets that will be dissociated", which causes concern about Hubs and VNets becoming disconnected.

The other concern is about having a disassociated Policy object in an orphaned state, and then the CAF Terraform module attempting to create a new Firewall Policy (with the same name), and causing a conflict. This did not occur though.

Durations

Manually removing the Policy association completed in approximately 5 minutes. Applying the CAF Terraform module (with the updated SKU), completed in approximately 20 minutes. Re-applying the Azure Firewall Policy with the Rule Collection Groups, etc. completed in approximately 8 minutes (though this is subjective to the number of Collection Groups / Rules that need to be created/applied).

[AErmie]

The firewall has been upgrade to Premium only in the FORGE environment, to allow for testing and investigating (before it will potentially be enabled in LIVE).