[aws_autoscaling_group] Provider produced inconsistent final plan (for tags)

yorinasub17 commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

%~> terraform version
Terraform v0.12.25
+ provider.aws v2.69.0
+ provider.random v2.3.0

Also tested and repro'd with:

terraform v0.12.28
terraform v0.13.0-beta3

Affected Resource(s)

aws_autoscaling_group

Terraform Configuration Files

provider "aws" {
  region = "us-east-1"
}

variable "ami_id" {
  # Ubuntu 16.04 image in us-east-1
  default = "ami-0fcfd45b96222a2ae"
}

# Set up autoscaling group

resource "aws_autoscaling_group" "web" {
  max_size             = 1
  min_size             = 1
  desired_capacity     = 1
  name                 = aws_launch_configuration.web_conf.name
  launch_configuration = aws_launch_configuration.web_conf.name
  vpc_zone_identifier  = data.aws_subnet_ids.default.ids

  tags = [
    {
      key                 = "Name"
      value               = "servers"
      propagate_at_launch = true
    },
    {
      key                 = "LCNAME1"
      value               = aws_launch_configuration.web_conf.name
      propagate_at_launch = true
    },
    {
      key                 = "LCNAME2"
      value               = aws_launch_configuration.web_conf.name
      propagate_at_launch = true
    },
  ]
}

resource "aws_launch_configuration" "web_conf" {
  name_prefix   = "web"
  image_id      = var.ami_id
  instance_type = "t2.micro"
}

# Look up default VPC and subnet to deploy server into

data "aws_vpc" "default" {
  default = true
}

data "aws_subnet_ids" "default" {
  vpc_id = data.aws_vpc.default.id
}

Expected Behavior

Apply completes successfully, creating the Autoscaling Group.

Actual Behavior

The first apply fails with the following error message:

Error: Provider produced inconsistent final plan
When expanding the plan for aws_autoscaling_group.web to include new values
learned so far during apply, provider "registry.terraform.io/-/aws" produced
an invalid new value for .tags: length changed from 2 to 3.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Note that the second apply succeeds, presumably because the values for all the tags are known at this point.

Steps to Reproduce

terraform init
terraform apply

yorinasub17 commented 4 years ago

NOTE: I have tried rolling back the provider versions, and the first known version that works is v2.63.0. v2.64.0 - v2.69.0 have all failed with the same error.

yorinasub17 commented 4 years ago

After comparing the debug logs between v2.63.0 and v2.64.0, I noticed the following difference:

The relevant log starts with:

2020/07/07 21:55:57 [WARN] Provider "registry.terraform.io/-/aws" produced an invalid plan for aws_autoscaling_group.web, but we are tolerating it because it
is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:

v2.63.0


- .tags: planned value
cty.ListVal(
  []cty.Value{
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("servers")}),
    cty.UnknownVal(cty.Map(cty.String)),
    cty.UnknownVal(cty.Map(cty.String))
  }
)
does not match config value
cty.ListVal(
  []cty.Value{
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("servers")}),
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("LCNAME1"), "propagate_at_launch":cty.StringVal("true"), "value":cty.UnknownVal(cty.String)}),
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("LCNAME2"), "propagate_at_launch":cty.StringVal("true"), "value":cty.UnknownVal(cty.String)}),
  }
)

v2.64.0

- .tags: planned value
cty.SetVal(
  []cty.Value{
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("servers")}),
    cty.UnknownVal(cty.Map(cty.String)),
  }
)
does not match config value
cty.SetVal(
  []cty.Value{
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("LCNAME1"), "propagate_at_launch":cty.StringVal("true"), "value":cty.UnknownVal(cty.String)}),
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("LCNAME2"), "propagate_at_launch":cty.StringVal("true"), "value":cty.UnknownVal(cty.String)}),
    cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("servers")}),
  }
)

So it appears that the transition to using sets for the tags has caused an issue with tags that depend on other resource values that are not known at plan time because those unknown vals (cty.UnknownVal(cty.String)) get deduped and are collapsed down.

yorinasub17 commented 4 years ago

Apologies for the spam - I deleted the message below when I thought the workaround didn't work when I integrated it into our real modules, but that was just a misunderstanding of the error message.

Restored message:

Ok I found a workaround by using tag blocks instead of tags attribute:

resource "aws_autoscaling_group" "web" {
  max_size             = 1
  min_size             = 1
  desired_capacity     = 1
  name                 = aws_launch_configuration.web_conf.name
  launch_configuration = aws_launch_configuration.web_conf.name
  vpc_zone_identifier  = data.aws_subnet_ids.default.ids

  dynamic "tag" {
    for_each = local.tags
    content {
      key                 = tag.value.key
      value               = tag.value.value
      propagate_at_launch = tag.value.propagate_at_launch
    }
  }
}

locals {
  tags = [
    {
      key                 = "Name"
      value               = "servers"
      propagate_at_launch = true
    },
    {
      key                 = "LCNAME1"
      value               = aws_launch_configuration.web_conf.name
      propagate_at_launch = true
    },
    {
      key                 = "LCNAME2"
      value               = aws_launch_configuration.web_conf.name
      propagate_at_launch = true
    },
  ]
}

anGie44 commented 4 years ago

hi @yorinasub17, thank you for reporting this issue and working through an alternative here! making a quick pass through your config, I can see the references to aws_launch_configuration.web_conf.name in the value key of the tags list could be the culprit. From the logs in your previous comment, terraform picks up only on the first tag where value is a literal quoted string "servers" but for the 2 subsequent tags which have expressions intended to be a references, they seemed to be ignored . Wrapping the expressions with parentheses, however, enables terraform to pick up on all 3 tags at plan time :) Example:

{
      key                 = "LCNAME1"
      value               = (aws_launch_configuration.web_conf.name)
      propagate_at_launch = true
    },
...

Snippet of plan w/original config

+ tags                      = [
          + {
              + "key"                 = "Name"
              + "propagate_at_launch" = "true"
              + "value"               = "servers"
            },
          + (known after apply),
        ]
      + target_group_arns         = (known after apply)

vs. Snippet of plan w/original config + wrapping exprs. w/parens:

 + tags                      = [
          + {
              + "key"                 = "LCNAME1"
              + "propagate_at_launch" = "true"
              + "value"               = "web54564564564"
            },
          + {
              + "key"                 = "LCNAME2"
              + "propagate_at_launch" = "true"
              + "value"               = "web54654654564"
            },
          + {
              + "key"                 = "Name"
              + "propagate_at_launch" = "true"
              + "value"               = "servers"
            },
        ]

yorinasub17 commented 4 years ago

@anGie44 Thanks for taking the time to look at this, and apologies for taking time to respond.

I just tried what you provided (with the parens), and unfortunately I still have the same issue. Looking at your plan, it looks like you tried the apply with parens on a second run based on the fact that the plan has the value resolved. In the initial run, those values should be unknowable because the launch configuration is not created yet.

Can you try running destroy to clear the state and then running plan with the parens? You should get the same plan as without parens in the initial run.

github-actions[bot] commented 4 months ago

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

dezren39 commented 3 months ago

still an issue

hashicorp / terraform-provider-aws