aws_wafv2_web_acl resource dependency wrong way around

[waqarkhan3389]

Terraform Version: 0.13.5 AWS provider version: v3.27.0 I have the following TF, when I run this after having run it before with a list of IP addresses it tries to delete the IP set before it removes the rule which was created including that IP set. I am unsure if this is due to me adding the for_each and making it a dynamic block though. In that case which config is correct? It results in the following error:

Error deleting WAFv2 IPSet: WAFAssociatedItemException: AWS WAF couldn’t perform the operation because your resource is being used by another resource or it’s associated with another resource.

resource "aws_wafv2_web_acl" "waf" {
  name        = "${local.name}-${var.environment}-WAF"
  description = "${local.name}-${var.environment}-WAF"
  scope       = "REGIONAL"

  default_action {
    allow {}
  }

  dynamic "rule" {
    for_each = var.waf_whitelist_ips != null ? [1] : []
    content {

           name     = "WAFWhitelistIPs"
           priority = 0

           action {
               allow {}
            }

           statement {

                  ip_set_reference_statement {
                   arn = aws_wafv2_ip_set.ipset[0].arn
                }
            }

           visibility_config {
               cloudwatch_metrics_enabled = true
               metric_name                = "${local.name}-${var.environment}-WHITELISTIps"
               sampled_requests_enabled   = true
            }
        }
  }

  rule {
    name     = "AWSManagedIpReputation"
    priority = 1

    override_action {
      none {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAmazonIpReputationList"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "${local.name}-${var.environment}-WAF-IP-Bad-Rep"
      sampled_requests_enabled   = true
    }
  }

// WAF whitelist IP set
resource "aws_wafv2_ip_set" "ipset" {
  count = var.waf_whitelist_ips != null ? 1 : 0
  name = "WAFWhitelistIPs"
  scope = "REGIONAL"
  ip_address_version = "IPV4"
  addresses = var.waf_whitelist_ips
}

[anGie44]

Hi @waqark3389 , thank you for creating this issue. I'm unable to reproduce on my end unfortunately with the complete config you've provided, the dynamic block seems to work as I'd expect. Would you be able to provide any debug logs to further investigate the error?

[paliwalvimal]

I facing the same issue. The problem isn't with dynamic block here. Instead, when you will set waf_whitelist_ips to null it won't be able to delete the IP set as it is being used within the rule. hence, until the rule is removed from the rule IP set cannot be deleted

[ZF-fredericvanlinthoudt]

We are facing this issue when trying to remove a rule that is referencing an ip set. It first tries to destroy the IP set but that does not work because it is still being referenced in the web acl rule. The actual behaviour should be to first adjust the web acl and delete the necessary rule and then destroying the ip set. Also settings an explicit depends_on in the web acl to the ip set does not solve the problem.

[boxrick]

To confirm, I have the same issue when I try and delete an ip ruleset it fails because it tries to delete it first without removing the ruleset first.

[gergelykovacs]

Hello! I am facing with the same issue (Terraform v0.15.1 + provider registry.terraform.io/hashicorp/aws v3.37.0). In order to reproduce:

1. Create e.g., 2 aws_wafv2_ip_set or aws_wafv2_regex_pattern_set and link in aws_wafv2_web_acl via rule. 1 acl with 2 rules.
2. Remove one aws_wafv2_ip_set or aws_wafv2_regex_pattern_set and the related rule.
3. Create plan: it should show that 1 rule and e.g., 1 ip set to be removed with the acl updated in place.

Result: Terraform starts to delete aws_wafv2_ip_set first not the rule which is not possible. Expected: Terraform should start to remove rule first.

The issue is the same if the resource name changes and acl rule to be updated. The lifecycle does not have an impact.

[Pola93]

Hello, I had similar issue when aws_wafv2_regex_pattern_set couldn't be destroyed as it was used by aws_wafv2_web_acl. I added depends_on = [aws_wafv2_regex_pattern_set.custom_allow_regex] to aws_wafv2_web_acl resource and it still did not help exactly like in cases described above.

What DID help was to add

lifecycle {
    create_before_destroy = true
  }

to both aws_wafv2_regex_pattern_set and aws_wafv2_web_acl resources so now terraform is firstly create new aws_wafv2_regex_pattern_set, update it in aws_wafv2_web_acl and then it is free to destroy previous aws_wafv2_regex_pattern_set resource. Hope it will help you :)

[relausen-tlg]

We're running into this problem too with 0.15.x. Does anyone know if it has been fixed in 1.x?

[phuocntsts]

Hello, I had similar issue when aws_wafv2_regex_pattern_set couldn't be destroyed as it was used by aws_wafv2_web_acl. I added depends_on = [aws_wafv2_regex_pattern_set.custom_allow_regex] to aws_wafv2_web_acl resource and it still did not help exactly like in cases described above.

What DID help was to add

lifecycle {
    create_before_destroy = true
  }

to both aws_wafv2_regex_pattern_set and aws_wafv2_web_acl resources so now terraform is firstly create new aws_wafv2_regex_pattern_set, update it in aws_wafv2_web_acl and then it is free to destroy previous aws_wafv2_regex_pattern_set resource. Hope it will help you :)

It worked for me. Thanks

[jmeridth]

@phuocntsts what version of the provider are you using? Original issue was filed with v3.27.0. Latest is v3.63.0. v3.47.0 was latest release when @Pola93 claimed the fix you're using. So has this been fixed since v3.47.0? Possibly fixed by #17876 that was included in release v3.33.0

[phuocntsts]

@phuocntsts what version of the provider are you using? Original issue was filed with v3.27.0. Latest is v3.63.0. v3.47.0 was latest release when @Pola93 claimed the fix you're using. So has this been fixed since v3.47.0? Possibly fixed by #17876 that was included in release v3.33.0

Hi @jmeridth , my current version is v3.62.0. today I get the same error and it worked for me when I try with "lifecycle".

[Shuvo-Hoque]

Hi, I also solved exactly this problem using "lifecycle" block. I was trying to modify ip set, regex pattern set and rule groups. Thank you.

lifecycle {
    create_before_destroy = true
  }

[RomanDanyk]

I tried to use the lifecycle block but Terraform expectedly failed to create a WAF rule group (in my case) because of the already existing one. We use Terraform v1.0.0 and terraform-provider-aws v4.15.1.

[nickeeromo]

Disregard - I can't delete this comment

[lorelei-rupp-imprivata]

We are hitting this too, in our case the create_before_destroy does not work because the existing waf is just being updated not replaced

To workaround this we had to do a two step decomm First update the waf to remove the rules that no longer use the other resources. Then apply that Then we can decome the regex/ipset resources in a second apply

[azt3k]

Still a problem when updating an existing WAF with Terraform 1.3.2 and AWS provider 4.56.0.

Lifecycle block did not help.

[olivatooo]

Still having this problem :/

[popsicleslayer]

It is still a problem when updating WAF: Terraform v1.4.6 and AWS provider 4.66.1

[ankon]

For anyone hitting this, and neither being able to fix it with the create_before_destroy configuration nor being able to temporarily remove resources to make TF happy: Another option that now worked for me is to manually remove the resources that TF wants to delete from the state (using terraform state rm), let TF do the other updates, and then manually remove the resources using the AWS console/CLI.

[fdutheil]

TF still can't handle this simple dependency correctly, failing horribly as an IaC tool (manual workarounds, lol). I can't wait for this issue to turn 3 year-old (I collect TF old issues that never get fixed because they are deeply rooted into conceptions flaws. That's why I'm in such a good mood).

[msonowal]

where should we add in the waf resource or rule resource ?

lifecycle {
    create_before_destroy = true
  }

as I create dynamic rules like this

 resource "aws_wafv2_ip_set" "waf_ip_sets" {
  for_each           = local.clientIpSets
  name               = "${var.infra_env}-${lower(each.key)}-ips"
  ip_address_version = each.value.version
  addresses          = each.value.addresses
  description        = each.value.description
  scope              = "REGIONAL"
  tags = {
    CreatedBy = "Manash"
    ManagedBy = "terraform"
    infra_env = var.infra_env
  }

}```

[air3ijai]

where should we add in the waf resource or rule resource ?

lifecycle {
    create_before_destroy = true
  }

as I create dynamic rules like this

resource "aws_wafv2_ip_set" "waf_ip_sets" {
 for_each           = local.clientIpSets
 name               = "${var.infra_env}-${lower(each.key)}-ips"
 ip_address_version = each.value.version
 addresses          = each.value.addresses
 description        = each.value.description
 scope              = "REGIONAL"
 tags = {
   CreatedBy = "Manash"
   ManagedBy = "terraform"
   infra_env = var.infra_env
 }

}```

As it is mentioned in the comment

lifecycle {
    create_before_destroy = true
}

should be added to both resources

• aws_wafv2_web_acl
• aws_wafv2_ip_set

You can try this step by step. First add to aws_wafv2_web_acl and if not fixed, add it to aws_wafv2_ip_set and check.

[zamirTo1]

Adding a lifecycle does not fix the issue.

lifecycle {
    create_before_destroy = true
}

[air3ijai]

What is your case and components versions?

[ckreiling]

We are also hitting this scenario, but in our case it is an IPSet referenced in a RuleGroup. Sadly using lifecycle/depends_on blocks doesn't work because the RuleGroup is just updated, not recreated. Not to mention that RuleGroups themselves are referenced in web ACLs, so forcing recreation wouldn't work either.

It would be greatly appreciated if this was addressed officially, even if the fix is simply a well-defined workaround instead of code changes in this provider. Web ACLs are critical pieces of security infrastructure, thus it's very important to have the ability to remove IPSets (and thus their references) from infrastructure swiftly and confidently

edit: we are on hashicorp/aws v5.22.0, Terraform ~>1.6

[ashwinSALGAOCAR]

Hello, I had similar issue when aws_wafv2_regex_pattern_set couldn't be destroyed as it was used by aws_wafv2_web_acl. I added depends_on = [aws_wafv2_regex_pattern_set.custom_allow_regex] to aws_wafv2_web_acl resource and it still did not help exactly like in cases described above.

What DID help was to add

lifecycle {
    create_before_destroy = true
  }

to both aws_wafv2_regex_pattern_set and aws_wafv2_web_acl resources so now terraform is firstly create new aws_wafv2_regex_pattern_set, update it in aws_wafv2_web_acl and then it is free to destroy previous aws_wafv2_regex_pattern_set resource. Hope it will help you :)

Thank you @Pola93. This worked for me. To provide more context for my case: terraform version: 1.4.6 aws provider version: 5.15.0

I was renaming the aws_wafv2_ip_set and aws_wafv2_rule_group resources previously created. So terraform apply was destroying them before updating the associated aws_wafv2_web_acl rules. To fix this dependency, I only added the lifecycle { create_before_destroy = true } block to the aws_wafv2_ip_set and aws_wafv2_rule_group resource blocks and it worked on terraform apply.

Note: _I did not have to add the lifecycle { create_before_destroy = true } block to aws_wafv2_web_acl_.

[Nolan-Amouyal]

The create_before_destroy didn't work for me.

I was able to make it work for me by forcing Terraform to replace the aws_wafv2_web_acl resource (instead of update in-place), using this:

resource "terraform_data" "web_acl_replacement" {
  input = var.variable_that_triggers_ipset_destruction
}

# and in the aws_wafv2_web_acl:
lifecycle {
    replace_triggered_by = [terraform_data.web_acl_replacement]
}

Now if the ip set must be destroyed, it will also destroy the web acl. In this case it DOES respect the dependency and destroys the web acl BEFORE the ip set. I just don't understand why the order is not respected for update in-place.

However this may apply some additional costs since waf web acls are paid resources on aws (do you pay for 2 acls if you destroy one and create a second one in a given month?)

[air3ijai]

However this may apply some additional costs since waf web acls are paid resources on aws (do you pay for 2 acls if you destroy one and create a second one in a given month?)

AWS WAF Pricing

You will be charged for each web ACL that you create and each rule that you create per web ACL. In addition, you will be charged for the number of web requests processed by the web ACL. Pricing may vary across AWS Regions. Monthly fees are prorated hourly. Pricing for AWS WAF Classic is the same as shown in the table below.

Web ACL | $5.00 per month (prorated hourly) Rule | $1.00 per month (prorated hourly)

1 WAF + 10 Rule = 1 x 5 + 10 x 1 = 15$/m ~ 0,02$/h (15/730)

Which resources do you use? In our case we use a CloudFront distribution and a simple WAF ACL and it works as per workaround

aws_cloudfront_distribution

aws_wafv2_web_acl + lifecycle {create_before_destroy = true}

• plan --> update CloudFront distribution - applied
• lifecycle --> create new WAF - not required
• plan --> destroy existing WAF - applied

Terraform v1.9.3 + hashicorp/aws v5.60.0

[mleziva]

Facing the same issue. Perhaps the maintainers of the aws provider could add a new resource specifically for managing and deploying ACL rules

[SafeEval]

Also running into this issue with wafv2_web_acl and wafv2_rule_group.

This reminds me of the issue with provisioning AWS security groups in Terraform. It used to be that security group rules could only be defined in security_group resources with inline ingress and egress blocks. Multiple AWS resources cominbed in a single overloaded Terraform resource. Then the introduction of vpc_security_group_ingress_rule and vpc_security_group_egress_rule allowed more granular control of resources and getting around the creation order. It's more verbose, but also allows more precise and avoids these sorts of issues.

[Nolan-Amouyal]

Which resources do you use?

• aws_wafv2_ip_set for IPv4 whitelisting in the ACL
• aws_wafv2_web_acl with rules on the IP set, rate limit, geo whitelisting

x2 because there is one for Cloudfront and one for the regional load balancers.

There were lifecycle problems with the IP set whenever I wanted to destroy it (disable it using count on the resource and a variable).

If I remember correctly, it tried to destroy the IP Set before updating the ACL (removing the rule) and AWS does not allow IP set destruction if it is referenced in an ACL rule somewhere.

Setting the replace_triggered_by forced the ACL to be replaced instead of just updated in place, which (for some reason) happened in the correct order this time: it replaces the ACL first, and then destroys the IP set when it is not referenced anywhere anymore.

I set it on my variable so that everytime we want to change the "existence" of the IP set (enabling or disabling the whitelist rule), the ACL is forced replaced.

Using create_before_destroy didn't help at all in my use case.

[bcmedeiros]

Just had the same issue with aws_wafv2_web_acl and aws_wafv2_ip_set, I deleted the IP set and the rule from the ACL, and terraform is trying to delete the IP set before removing the rule.

[grasbergerm]

For anyone still running into this, I was able to get it to work when adding create_before_destroy to the aws_wafv2_web_acl BUT only when create_before_destroy is added before an aws_wafv2_ip_set is associated to the aws_wafv2_web_acl.

In my case, after the aws_wafv2_ip_set is in use, I was not able to get Terraform to recognize that the aws_wafv2_web_acl needed to be modified first in order to delete the aws_wafv2_ip_set. I manually removed the aws_wafv2_ip_set from the aws_wafv2_web_acl in the AWS Console and re-ran my apply, which then successfully destroyed the aws_wafv2_ip_set.

Then I added

lifecycle {
    create_before_destroy = true
  }

to the aws_wafv2_web_acl.

Next, I added back the aws_wafv2_ip_set, successfully ran an apply, then removed it and ran another apply, in which it properly modifies the aws_wafv2_web_acl first and the aws_wafv2_ip_set is able to be deleted.

Just to make things clear, here's the output from the last apply to destroy the aws_wafv2_ip_set:

14:32:10.202 module.cloudfront[0].aws_wafv2_web_acl.default: Modifying... [id=xxx]
14:32:11.539 module.cloudfront[0].aws_wafv2_web_acl.default: Modifications complete after 2s [id=xxx]
14:32:11.540 module.cloudfront[0].aws_wafv2_ip_set.ip_block_list[0]: Destroying... [id=xxx]
14:32:11.746 module.cloudfront[0].aws_wafv2_ip_set.ip_block_list[0]: Destruction complete after 0s