Circular Tag Clobbering with tags_all metadata resulting in tag overwrites

[bpr-git]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

• terraform version 0.14.11
• aws provider version 3.38.0 and up
• This does not happen with aws provider version 3.37.0

Affected Resource(s)

• tags metadata
• tags_all metadata

Terraform Configuration Files

# Tell ACM to use the above CNAME to validate the cert.
# Module ./local_modules/cert_and_validation
resource "aws_acm_certificate" "cert" {
  provider                  = aws.acm
  domain_name               = var.ssl_domain
  validation_method         = "DNS"
  subject_alternative_names = var.subject_alternative_names
  tags                      = var.cert_tags

  lifecycle {
    create_before_destroy = true
  }
}

# Configuration calling above module
module "api_cert" {
  source = "./local_modules/cert_and_validation"
  providers = {
    aws.acm     = aws
    aws.route53 = aws.nunahealth
  }
  ssl_domain                 = local.api_hostname
  subject_alternative_names  = var.subject_alternative_names
  cert_tags                  = merge(module.label.tags, module.nuna-label.tags)
  route53_validation_zone_id = local.verification_zone_id
}

Expected Behavior

When updating a tag in the configuration files:

1. terraform plan should show this change
2. terraform apply should apply this change
3. terraform plan should no longer show this change

Actual Behavior

1. terraform plan shows this change
2. terraform apply applies this change
3. terraform plan continues to show this change <-- This is the error!

Steps to Reproduce

1. Update a tag in HCL

2. terraform plan shows the change

   
   Terraform will perform the following actions:
   
   # module.api_cert.aws_acm_certificate.cert will be updated in-place
   ~ resource "aws_acm_certificate" "cert" {
       id                        = <redacted>
     ~ tags                      = {
         ~ "storage"          = "rds" -> ""
           # (14 unchanged elements hidden)
       }
       # (8 unchanged attributes hidden)
   
       # (1 unchanged block hidden)
   }

Plan: 0 to add, 1 to change, 0 to destroy.

3. `terraform state show` shows the following:

module.api_cert.aws_acm_certificate.cert:

resource "aws_acm_certificate" "cert" {

tags = { "storage" = "rds" } tags_all = { "storage" = "rds" } } ``` 4. `terraform apply` the plan ``` Apply complete! Resources: 0 added, 1 changed, 0 destroyed. ``` 5. **[This is where behavior becomes erroneous]** `terraform state show` does not reflect this change: ``` # module.api_cert.aws_acm_certificate.cert: resource "aws_acm_certificate" "cert" { id = tags = { "storage" = "rds" } tags_all = { "storage" = "rds" } } ``` 6. `terraform plan` again and we're back where we started ``` Terraform will perform the following actions: # module.api_cert.aws_acm_certificate.cert will be updated in-place ~ resource "aws_acm_certificate" "cert" { id = ~ tags = { ~ "storage" = "rds" -> "" # (14 unchanged elements hidden) } # (8 unchanged attributes hidden) # (1 unchanged block hidden) } Plan: 0 to add, 1 to change, 0 to destroy. ``` ### Important Factoids * This does not happen with aws provider version 3.37.0

[ericksoen]

A colleague and I experienced a similar issue to the one you described. The detail about the unique behaviors depending on the AWS provider version was super helpful 🎉. My colleague experienced the issue with hashicorp/aws v3.3.37 and I did not experience the issue with hashicorp/aws v3.46.0.

We were able to confirm that by executing terraform init --upgrade to bump their hashicorp/aws version to 3.3.48 resolved the issue.

Seems like there's some interesting behavior with tags_all starting with version 3.37.0 and persistent until 3.46.0 (at least from our minimal attempts to reproduce).

[ewbankkit]

Hey all 👋 Thank you very much for taking the time to raise this! This was addressed with #29747, which was included in version 5.0.0 of the provider. With that in mind, we'll close this issue. If you experience additional issues with the provider, please do open a new issue to let us know.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.