hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.62k stars 9k forks source link

[Bug]: WAF dissociate the ALB #34823

Open xyfleet opened 7 months ago

xyfleet commented 7 months ago

Terraform Core Version

Terraform v1.5.6

AWS Provider Version

5.30

Affected Resource(s)

aws_wafv2_web_acl_association

Expected Behavior

All of ALBs are associated with the WAF

Actual Behavior

One of ALBs got dissociated accidentally. Once you put it back through terraform apply, this issue will happen again. In general the dissociation will happen time by time. You have to run terraform apply to update the WAF association when you see it. This will bring security risk to us since the ALB got dissociated from WAF.

Relevant Error/Panic Output Snippet

resource "aws_wafv2_web_acl_association" "aws_alb" {
  for_each = toset(var.aws_lb_arn_list)

  resource_arn = each.value
  web_acl_arn  = aws_wafv2_web_acl.webacl.arn

}

Terraform Configuration Files

resource "aws_wafv2_web_acl_association" "aws_alb" {
  for_each = toset(var.aws_lb_arn_list)

  resource_arn = each.value
  web_acl_arn  = aws_wafv2_web_acl.webacl.arn

}

I put the ARNs of three ALBs into a list as a variable:

aws_lb_arn_list = ["arn:aws:elasticloadbalancing:xxxxxxxxxxx,
"arn:aws:elasticloadbalancing:yyyyyyyyyyy",
"arn:aws:elasticloadbalancing:zzzzzzzzzzz"
]

Steps to Reproduce

1: follow the configuration 2: and wait 3: someday, one of ALBs will be dissociated without any notification or code changing.

Debug Output

No response

Panic Output

No response

Important Factoids

When one ALB got dissociated, I checked the terraform state file, all of ALBs are listed there(I am sure there is no one to change this code). And when you run terraform plan, it will tell you that one association will be created. Once you run terraform apply this change, you will see the ALB got associated on AWS WAF console.

Below is the state file when I found the ALB got dissociated.

"module": "module.aws_waf",
      "mode": "managed",
      "type": "aws_wafv2_web_acl_association",
      "name": "aws_alb",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": "arn:aws:elasticloadbalancing:xxxxxxxxxxxxxxxx",
          "schema_version": 0,
          "attributes": {
            "id": "xxxxxxxxxxxxxxxx",
            "resource_arn": "arn:aws:elasticloadbalancing:xxxxxxxxxxxxxxxx",
            "timeouts": null,
            "web_acl_arn": "arn:aws:wafv2:ooooooooooooooooooo"
          },
         ... ...
        },
        {
          "index_key": "arn:aws:elasticloadbalancing:yyyyyyyyyyyyyyyy",
          "schema_version": 0,
          "attributes": {
            "id": "yyyyyyyyyyyyyyyy",
            "resource_arn": "arn:aws:elasticloadbalancing:yyyyyyyyyyyyyyyy",
            "timeouts": null,
            "web_acl_arn": "arn:aws:wafv2:ooooooooooooooooooo"
          },
         ... ...
        },
        {
          "index_key": "arn:aws:elasticloadbalancing:zzzzzzzzzzzzzzz",
          "schema_version": 0,
          "attributes": {
            "id": "zzzzzzzzzzzzzzz",
            "resource_arn": "arn:aws:elasticloadbalancing:zzzzzzzzzzzzzzz",
            "timeouts": null,
            "web_acl_arn": "arn:aws:wafv2:ooooooooooooooooooo"
          },
          ... ...
          ]
        }
      ]
    },

I checked the log from CloudTrail and found that the DisassociateWebACL command was triggered via the ALB SDK as per the user agent.

I am not quiet sure why this command got triggered. How it was triggered. Any idea will be appreciated.

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 7 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

acwwat commented 6 months ago

This seems unrelated to the Terraform configuration since the disassociation happens after the configuration is applied successfully. The CloudTrail record that you mention should have more details to help determine the source. I would recommend further checking the userIdentity, sourceIPAddress, and userAgent attributes.