error deleting NetworkFirewall Rule Group because it is still in use

[ADF1111]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Hello everyone,

I am deploying AWS network firewall using terraform. I am experiencing an error when trying to delete rule groups. It seems that the rule group is not first deleted from the network firewall policy, so terraform shows an error because the rule group is still being in used. Everything works fine if I go to the console and I delete the association between network firewall policy and the rule group. I think the problem is coming because I am using a local variable which value is a list of ARN's depending on the rules that will be created (I am using a module for network firewall deployment). I don't know how can I solve this dependency problem.

Terraform CLI and Terraform AWS Provider Version

Terraform v0.14.8

• provider registry.terraform.io/hashicorp/aws v3.47.0
• provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

• aws_networkfirewall_rule_group

Terraform Configuration Files

locals{
  create_any_stateful_rule_group = var.create_stateful_fw_rule_deny_access_to_domain || var.create_stateful_fw_rule_allow_IP_address || var.create_stateful_fw_rule_deny_destination || var.create_stateful_fw_rule_import_suricata
  fw_rule_deny_access_to_domain = var.create_stateful_fw_rule_deny_access_to_domain ? aws_networkfirewall_rule_group.deny_access_to_domain[0].arn : ""
  fw_rule_allow_access_to_domain = var.create_stateful_fw_rule_allow_access_to_domain ? aws_networkfirewall_rule_group.allow_access_to_domain[0].arn : ""
  fw_rule_allow_IP_address      = var.create_stateful_fw_rule_allow_IP_address ? aws_networkfirewall_rule_group.allow_IP_address[0].arn : ""
  fw_rule_deny_destination      = var.create_stateful_fw_rule_deny_destination ? aws_networkfirewall_rule_group.deny_destination.*.arn : [""]
  fw_rule_import_suricata      = var.create_stateful_fw_rule_import_suricata ? aws_networkfirewall_rule_group.suricata_file.*.arn : [""]
  network_fw_stateful_rule_group = concat([local.fw_rule_deny_access_to_domain,local.fw_rule_allow_access_to_domain,local.fw_rule_allow_IP_address])
  network_fw_stateful_rule_group_2 = concat(local.network_fw_stateful_rule_group,local.fw_rule_deny_destination)
  network_fw_stateful_rule_group_3 = concat(local.network_fw_stateful_rule_group_2,local.fw_rule_import_suricata)
  network_fw_stateful_rule_group_compact = compact(local.network_fw_stateful_rule_group_3)
}
resource "aws_networkfirewall_firewall" "network-firewall" {
  name                = "${var.vpc_name}-network-firewall"
  firewall_policy_arn = aws_networkfirewall_firewall_policy.network-firewall-policy.arn
  vpc_id              = var.vpc_id

  delete_protection   = var.fw_delete_protection
  description         = var.fw_description
  firewall_policy_change_protection   = var.fw_policy_change_protection
  subnet_change_protection            = var.fw_subnet_change_protection

  dynamic "subnet_mapping" {
    for_each  = [for s in var.subnet_fw_ids: {
      fw_subnet_id   = s
    }]
    content{
        subnet_id = subnet_mapping.value.fw_subnet_id
    }
  }

  tags = merge(
    var.tags,
    {
      "Name"    = "${var.vpc_name}-network-firewall"
    },
  )
}

resource "aws_networkfirewall_firewall_policy" "network-firewall-policy" {
  name = "${var.vpc_name}-network-firewall-policy"
  depends_on = [aws_networkfirewall_rule_group.allow_IP_address, aws_networkfirewall_rule_group.deny_destination, aws_networkfirewall_rule_group.suricata_file, aws_networkfirewall_rule_group.allow_access_to_domain, aws_networkfirewall_rule_group.deny_access_to_domain, aws_networkfirewall_rule_group.stateless_rule]

  firewall_policy {
    stateless_default_actions          = [var.fw_policy_stateless_default_actions]
    stateless_fragment_default_actions = [var.fw_policy_stateless_fragment_default_actions]

    dynamic "stateful_rule_group_reference" {
    for_each  = [for rule_grpup_arn in local.network_fw_stateful_rule_group_compact: {
      fw_stateful_rule_group   = rule_grpup_arn
    }]
    content{
        resource_arn  = stateful_rule_group_reference.value.fw_stateful_rule_group
    }
   }

   dynamic "stateless_rule_group_reference" {
    for_each = var.stateless_rule_group
    content{
      priority     = stateless_rule_group_reference.value.firewall_policy_priority
      resource_arn = aws_networkfirewall_rule_group.stateless_rule[stateless_rule_group_reference.key].arn
     }
    }

  }

  tags = merge(
    var.tags,
    {
      "Name"    = "${var.vpc_name}-network-firewall-policy"
    },
  )

}

resource "aws_networkfirewall_rule_group" "deny_access_to_domain" {
  count    = var.create_stateful_fw_rule_deny_access_to_domain ? 1 : 0
  capacity = 100
  description = "denying access to a domain"
  name     = var.stateful_fw_rule_deny_access_to_domain_name
  type     = "STATEFUL"
  rule_group {

    dynamic "rule_variables" {
      for_each =  length(var.stateful_fw_rule_deny_access_to_domain_home_net) > 0 ? [1] : []
      content {
        ip_sets {
          key = "HOME_NET"
          ip_set {
            definition = var.stateful_fw_rule_deny_access_to_domain_home_net
          }
        }
      }
    }
    rules_source {
      rules_source_list {
        generated_rules_type = "DENYLIST"
        target_types         = var.stateful_fw_rule_deny_access_to_domain_target_types
        targets              = var.stateful_fw_rule_deny_access_to_domain_targets
      }
    }
  }

  tags = var.tags
}

resource "aws_networkfirewall_rule_group" "stateless_rule" {
  for_each = var.stateless_rule_group
  capacity = each.value.capacity
  name     = each.value.name
  type     = "STATELESS"
  rule_group {
    rules_source {
      stateless_rules_and_custom_actions {
        dynamic "stateless_rule" { 
          for_each = [for k,v in each.value.rules: {
                  rule   = each.value.rules[k]
                  }]
          content{
            priority = stateless_rule.value.rule.priority 
            rule_definition {
              actions = stateless_rule.value.rule.actions
              match_attributes {
                source {
                  address_definition = lookup(stateless_rule.value.rule,"source_address",null)
                }
                dynamic "source_port" {
                  for_each = [for source in stateless_rule.value.rule.source_port: {
                  srcport   = source
                  }]
                  content{
                      from_port = source_port.value.srcport
                  }

                }
                destination {
                  address_definition = lookup(stateless_rule.value.rule,"destination_addres",null)
                }
                dynamic "destination_port" {
                  for_each =  [for destination in stateless_rule.value.rule.destination_port: {
                  dstport   = destination
                  }]
                  content{
                      from_port = destination_port.value.dstport
                  }

                }
                protocols = lookup(stateless_rule.value.rule,"protocols",null)
                dynamic "tcp_flag" {
                  for_each = [for tcp_flag in stateless_rule.value.rule.tcp_flags: {
                  tcp_flag   = tcp_flag
                  }]
                  content{
                      flags = tcp_flag.value.tcp_flag
                  }

                }
              }
            }
          }
        }
      }
    }
  }

    tags = var.tags

}

Expected Behavior

Rule group first deleted from the firewall policy and then deleted completely.

Actual Behavior

Error because rule group is still in use.

Steps to Reproduce

1. terraform apply

Important Factoids

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
  - destroy

Terraform will perform the following actions:

  # module.inspection-network-fw.aws_networkfirewall_firewall_policy.network-firewall-policy will be updated in-place
  ~ resource "aws_networkfirewall_firewall_policy" "network-firewall-policy" {
        id           = "arn:aws:network-firewall:eu-west-1:x:firewall-policy/inspection-vpc-network-firewall-policy"
        name         = "inspection-vpc-network-firewall-policy"
        tags         = {
            "Confidentiality" = "C2"
            "Environment"     = "dev"
            "ManagedBy"       = ""
            "Name"            = "inspection-vpc-network-firewall-policy"
            "Project"         = "network-firewall-offering"
        }
        # (3 unchanged attributes hidden)

      ~ firewall_policy {
            # (2 unchanged attributes hidden)

          - stateless_rule_group_reference {
              - priority     = 10 -> null
              - resource_arn = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-A-to-B" -> null
            }
          - stateless_rule_group_reference {
              - priority     = 1 -> null
              - resource_arn = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-B-to-A" -> null
            }
          + stateless_rule_group_reference {
              + priority     = 5
              + resource_arn = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-B-to-A-v2"
            }
          + stateless_rule_group_reference {
              + priority     = 7
              + resource_arn = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-A-to-B-v2"
            }
            # (3 unchanged blocks hidden)
        }
    }

  # module.inspection-network-fw.aws_networkfirewall_rule_group.stateless_rule["rule-1"] will be destroyed
  - resource "aws_networkfirewall_rule_group" "stateless_rule" {
      - arn          = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-B-to-A" -> null
      - capacity     = 10 -> null
      - id           = "arn:aws:network-firewall:eu-west-1:x:stateless-rulegroup/drop-spoke-B-to-A" -> null
      - name         = "drop-spoke-B-to-A" -> null
      - tags         = {
          - "Confidentiality" = "C2"
          - "Environment"     = "dev"
          - "ManagedBy"       = ""
          - "Project"         = "network-firewall-offering"
        } -> null
      - tags_all     = {
          - "Confidentiality" = "C2"
          - "Environment"     = "dev"
          - "ManagedBy"       = "adf@amazon.com"
          - "Project"         = "network-firewall-offering"
        } -> null
      - type         = "STATELESS" -> null
      - update_token = "39a0f475-1b37-4c44-af7c-4959bd2faebd" -> null

      - rule_group {

          - rules_source {

              - stateless_rules_and_custom_actions {

                  - stateless_rule {
                      - priority = 1 -> null

                      - rule_definition {
                          - actions = [
                              - "aws:drop",
                            ] -> null

                          - match_attributes {
                              - protocols = [] -> null

                              - destination {
                                  - address_definition = "10.1.0.0/16" -> null
                                }

                              - source {
                                  - address_definition = "10.2.0.0/16" -> null
                                }
                            }
                        }
                    }
                }
            }
        }
    }

  # module.inspection-network-fw.aws_networkfirewall_rule_group.stateless_rule["rule-2"] will be destroyed
  - resource "aws_networkfirewall_rule_group" "stateless_rule" {
      - arn          = "arn:aws:network-firewall:eu-west-1:299537281981:stateless-rulegroup/drop-spoke-A-to-B" -> null
      - capacity     = 10 -> null
      - id           = "arn:aws:network-firewall:eu-west-1:299537281981:stateless-rulegroup/drop-spoke-A-to-B" -> null
      - name         = "drop-spoke-A-to-B" -> null
      - tags         = {
          - "Confidentiality" = "C2"
          - "Environment"     = "dev"
          - "ManagedBy"       = "adf@amazon.com"
          - "Project"         = "network-firewall-offering"
        } -> null
      - tags_all     = {
          - "Confidentiality" = "C2"
          - "Environment"     = "dev"
          - "ManagedBy"       = "adf@amazon.com"
          - "Project"         = "network-firewall-offering"
        } -> null
      - type         = "STATELESS" -> null
      - update_token = "e9c5ccb3-ea43-4444-9870-e9ef31f4d3f5" -> null

      - rule_group {

          - rules_source {

              - stateless_rules_and_custom_actions {

                  - stateless_rule {
                      - priority = 1 -> null

                      - rule_definition {
                          - actions = [
                              - "aws:drop",
                            ] -> null

                          - match_attributes {
                              - protocols = [] -> null

                              - destination {
                                  - address_definition = "10.1.0.0/16" -> null
                                }

                              - source {
                                  - address_definition = "10.2.0.0/16" -> null
                                }
                            }
                        }
                    }
                }
            }
        }
    }

Plan: 0 to add, 1 to change, 2 to destroy.

References

• 0000

[anGie44]

Related https://github.com/hashicorp/terraform-provider-aws/issues/19616

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.