[Bug]: Version `5.x` of the Terraform AWS provider removes externally-managed tags from the `aws_autoscaling_group` resource when created by a module

[lisushka]

Terraform Core Version

1.5.2

AWS Provider Version

5.0.0, 5.14.0

Affected Resource(s)

aws_autoscaling_group

Expected Behavior

Externally-managed tags on ASGs should not be removed by Terraform

Actual Behavior

Externally-managed tags on ASGs are removed by Terraform

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

Minimum ASG module to reproduce:

data.tf:

data "aws_caller_identity" "current" {}

data "aws_arn" "current" {
  arn = data.aws_caller_identity.current.arn
}

main.tf:

resource "aws_autoscaling_group" "asg" {
  lifecycle {
    create_before_destroy = true
    ignore_changes        = [desired_capacity]
  }

  count                     = var.enabled ? 1 : 0
  name                      = var.asg_name
  max_size                  = var.asg_max
  min_size                  = var.asg_min == "" ? var.asg_max : var.asg_min
  desired_capacity          = var.asg_desired == "" ? var.asg_min == "" ? var.asg_max : var.asg_min : var.asg_desired
  force_delete              = false
  vpc_zone_identifier       = var.subnets
  termination_policies      = ["OldestInstance"]

  launch_template {
    id      = aws_launch_template.asg_launch_template[0].id
    version = "$Latest"
  }

  tag {
    key                 = "test-tag-01"
    value               = "test-value-01"
    propagate_at_launch = true
  }

  tag {
    key                 = "test-tag-02"
    value               = "test-value-02"
    propagate_at_launch = true
  }

  target_group_arns   = []
}

resource "aws_launch_template" "asg_launch_template" {
  count = var.enabled ? 1 : 0

  name          = var.asg_name
  image_id      = var.asg_ami
  ebs_optimized = true
  instance_type = var.asg_size

  iam_instance_profile {
    name = var.asg_iam_role
  }

  network_interfaces {
    associate_public_ip_address = true
    security_groups             = var.security_group_ids
    delete_on_termination       = true
  }

  metadata_options {
    instance_metadata_tags      = "enabled"
    http_endpoint               = "enabled"
    http_tokens                 = "required"
    http_put_response_hop_limit = 1
  }
}

variables.tf:

variable "asg_desired" {
  default = ""
}

variable "asg_max" {
}

variable "asg_name" {
}

variable "asg_min" {
  default = ""
}

variable "subnets" {
  type = list(string)
}

variable "asg_ami" {
}

variable "asg_size" {
}

variable "asg_iam_role" {
  default = "server"
}

variable "security_group_ids" {
  type = list(string)
}

variable "target_group_arns" {
  type    = list(string)
  default = []
}

variable "enabled" {
  type        = bool
  default     = true
}

State directory which calls the ASG module:

data.tf:

data "aws_caller_identity" "current" {}

asg.tf:

module "autoscaling" {
  source                  = # folder where the module code is
  subnets                 = # list of private subnet IDs for your VPC - we pass these in from pre-existing VPCs
  security_group_ids      = # list of security groups - we pass these in from pre-existing SGs

  asg_name                = "test-asg"
  asg_min                 = "0"
  asg_max                 = "0"
  asg_ami                 = # AMI to build instances in the ASG with - we pass in a pre-existing AMI
  asg_size                = "c5.large"
  asg_iam_role            = "server"
}

versions.tf:

terraform {
  required_version = "1.5.2"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.0.0"
    }
    local = {
      source  = "hashicorp/local"
      version = "1.4.0"
    }
  }
}

Steps to Reproduce

Stand up the ASG module, then manually add a tag to the auto-scaling group. Run a terraform plan - the new tag will be removed.

Debug Output

When planning the above code, after manually adding a tag to the ASG, in version 4.67.0:

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and
found no differences, so no changes are needed.

When planning the same directory on version 5.0.0 (the earliest version where we have observed this behaviour):

Terraform will perform the following actions:

  # module.autoscaling.aws_autoscaling_group.asg[0] will be updated in-place
  ~ resource "aws_autoscaling_group" "asg" {
        id                        = "test-asg"
        name                      = "test-asg"
        # (25 unchanged attributes hidden)

      - tag {
          - key                 = "test-tag-03" -> null
          - propagate_at_launch = true -> null
          - value               = "test-value-03" -> null
        }

        # (3 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

The externally-applied tags are also removed on version 5.14.0 (the newest version that we've tried).

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[hugo-barros]

Facing the same here and is a blocker for upgrading to provider v5.x for now. Have you found any solution to this?

[ewbankkit]

I have verified that behavior changed between v4.67.0 and v5.0.0 if there are configured tag values and additional tags are added external to Terraform, independent of the use of modules. With v4.x, no changes are shown. With v5.x, the externally added tags are to be removed.

Relates https://github.com/hashicorp/terraform-provider-aws/pull/30842.

[ewbankkit]

@lisushka / @hugo-barros You are correct that the behavior did change at v5.0.0 and the aws_autoscaling_group.tag attribute now behaves like other resources' tags attribute in that any manually/externally added tags will show as diffs in terraform plan and be removed from the AutoScaling Group if the plan is applied.

To work around this you can use a provider-level ignore_tags block:

provider "aws" {
  ...

  ignore_tags {
    keys = ["test-tag-03"]
  }
}

or add tag to lifecycle.ignore_changes:

resource "aws_autoscaling_group" "asg" {
  lifecycle {
    create_before_destroy = true
    ignore_changes        = [desired_capacity, tag]
  }

  ...
}