Bug during apply.

[fubarbaz]

See the generated attached data: terraform.txt

I haven't created a minimal example to reproduce this, so there is no point in asking.

How can I get the system into a clean state again?

[apparentlymart]

Hi @fubarbaz. Sorry for the issue here.

For this "diffs didn't match" class of error, often it is transient and a further run of Terraform will clear it. It's still a bug that we should fix, but from your perspective you should be able to re-run the plan/apply sequence and see it complete successfully.

The reason this works is that when you run terraform plan (or implicitly plan as part of running terraform apply) Terraform will refresh the state of the resource from the underlying API, thus repairing any inconsistencies.

[fubarbaz]

@apparentlymart It was not transient. I had to terraform destroy in order to get it in a decent state again. For production workloads terraform seems incredibly scary.

[jim3ma]

The same error

terraform plan:

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.

data.template_file.ecs_iam_policy: Refreshing state...
aws_cloudwatch_log_group.app: Refreshing state... (ID: tf-ecs-group/app-ghost)
aws_iam_role.ecs_service: Refreshing state... (ID: tf-example-ecs-role)
aws_ecs_cluster.main: Refreshing state... (ID: arn:aws:ecs:us-west-2:498302088937:cluster/tf-cads-ecs-cluster)
aws_vpc.main: Refreshing state... (ID: vpc-2c13de4b)
aws_iam_role.ec2_instance: Refreshing state... (ID: tf-ecs-example-instance-role)
aws_cloudwatch_log_group.ecs: Refreshing state... (ID: tf-ecs-group/ecs-agent)
data.aws_availability_zones.available: Refreshing state...
data.aws_ami.stable_coreos: Refreshing state...
aws_iam_instance_profile.app: Refreshing state... (ID: tf-ecs-instprofile)
aws_iam_role_policy.ecs_service: Refreshing state... (ID: tf-example-ecs-role:tf-example-ecs-policy)
data.template_file.task_definition: Refreshing state...
aws_ecs_task_definition.ghost: Refreshing state... (ID: tf-example-ghost-td)
data.template_file.ec2_iam_policy: Refreshing state...
aws_iam_role_policy.ec2_instance: Refreshing state... (ID: tf-ecs-example-instance-role:TfEcsExampleInstanceRole)
data.template_file.ec2_init_config: Refreshing state...
aws_subnet.main.0: Refreshing state... (ID: subnet-e0838296)
aws_alb_target_group.test: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-west-2:498302088937:targetgroup/tf-example-ecs-ghost/302ebd2ec1cd4661)
aws_security_group.lb_sg: Refreshing state... (ID: sg-3586a94c)
aws_internet_gateway.gw: Refreshing state... (ID: igw-6537ae01)
aws_subnet.main.1: Refreshing state... (ID: subnet-f54ea192)
aws_security_group.ec2_sg: Refreshing state... (ID: sg-1986a960)
aws_route_table.r: Refreshing state... (ID: rtb-4e34a229)
aws_alb.main: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-west-2:498302088937:loadbalancer/app/tf-example-alb-ecs/42515b2465fbea20)
aws_route_table_association.a.0: Refreshing state... (ID: rtbassoc-7c1cd51a)
aws_route_table_association.a.1: Refreshing state... (ID: rtbassoc-7f1cd519)
aws_launch_configuration.app: Refreshing state... (ID: terraform-20161125015106574914313hge)
aws_autoscaling_group.app: Refreshing state... (ID: tf-cads-asg)
aws_alb_listener.front_end: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-west-2:498302088937:listener/app/tf-example-alb-ecs/42515b2465fbea20/1d160514347c4b75)
aws_ecs_service.test: Refreshing state... (ID: arn:aws:ecs:us-west-2:498302088937:service/tf-example-ecs-ghost)

The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

~ aws_autoscaling_group.app
    launch_configuration: "terraform-20161125015106574914313hge" => "<computed>"

-/+ aws_launch_configuration.app
    associate_public_ip_address: "true" => "true"
    ebs_block_device.#:          "0" => "<computed>"
    ebs_optimized:               "false" => "<computed>"
    enable_monitoring:           "true" => "true"
    iam_instance_profile:        "tf-ecs-instprofile" => "tf-ecs-instprofile"
    image_id:                    "ami-6f1eb80f" => "ami-6f1eb80f"
    instance_type:               "t2.small" => "t2.small"
    key_name:                    "iwsh-ecs-dev" => "iwsh-ecs-dev"
    name:                        "terraform-20161125015106574914313hge" => "<computed>"
    root_block_device.#:         "0" => "<computed>"
    security_groups.#:           "1" => "<computed>" (forces new resource)
    user_data:                   "7b812c1a2fe29401bb4491ae92ca6530b22a0ce1" => "7b812c1a2fe29401bb4491ae92ca6530b22a0ce1"

-/+ aws_security_group.ec2_sg
    description:                                  "controls direct access to application instances" => "controls direct access to application instances"
    egress.#:                                     "1" => "1"
    egress.482069346.cidr_blocks.#:               "1" => "1"
    egress.482069346.cidr_blocks.0:               "0.0.0.0/0" => "0.0.0.0/0"
    egress.482069346.from_port:                   "0" => "0"
    egress.482069346.prefix_list_ids.#:           "0" => "0"
    egress.482069346.protocol:                    "-1" => "-1"
    egress.482069346.security_groups.#:           "0" => "0"
    egress.482069346.self:                        "false" => "false"
    egress.482069346.to_port:                     "0" => "0"
    ingress.#:                                    "2" => "2"
    ingress.2701185744.cidr_blocks.#:             "1" => "1"
    ingress.2701185744.cidr_blocks.0:             "43.230.88.117/32" => "43.230.88.117/32"
    ingress.2701185744.from_port:                 "22" => "22"
    ingress.2701185744.protocol:                  "tcp" => "tcp"
    ingress.2701185744.security_groups.#:         "0" => "0"
    ingress.2701185744.self:                      "false" => "false"
    ingress.2701185744.to_port:                   "22" => "22"
    ingress.864464978.cidr_blocks.#:              "0" => "0"
    ingress.864464978.from_port:                  "8080" => "8080"
    ingress.864464978.protocol:                   "tcp" => "tcp"
    ingress.864464978.security_groups.#:          "1" => "1"
    ingress.864464978.security_groups.3156370434: "sg-3586a94c" => "sg-3586a94c"
    ingress.864464978.self:                       "false" => "false"
    ingress.864464978.to_port:                    "8080" => "8080"
    name:                                         "tf-ecs-instsg" => "tf-ecs-inst-sg" (forces new resource)
    owner_id:                                     "498302088937" => "<computed>"
    vpc_id:                                       "vpc-2c13de4b" => "vpc-2c13de4b"

terraform apply

Error applying plan:

1 error(s) occurred:

* aws_security_group.ec2_sg: diffs didn't match during apply. This is a bug with Terraform and should be reported as a GitHub Issue.

Please include the following information in your report:

    Terraform Version: 0.7.12
    Resource ID: aws_security_group.ec2_sg
    Mismatch reason: diff: Destroy; old: false, new: true
    Diff One (usually from plan): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"ingress.864464978.security_groups.3156370434":*terraform.ResourceAttrDiff{Old:"sg-3586a94c", New:"sg-3586a94c", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.security_groups.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.security_groups.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.from_port":*terraform.ResourceAttrDiff{Old:"22", New:"22", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.to_port":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.prefix_list_ids.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.cidr_blocks.0":*terraform.ResourceAttrDiff{Old:"43.230.88.117/32", New:"43.230.88.117/32", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "vpc_id":*terraform.ResourceAttrDiff{Old:"vpc-2c13de4b", New:"vpc-2c13de4b", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "owner_id":*terraform.ResourceAttrDiff{Old:"498302088937", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.protocol":*terraform.ResourceAttrDiff{Old:"tcp", New:"tcp", NewComputed:false, NewRemoved:false, NewExtra:"tcp", RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "name":*terraform.ResourceAttrDiff{Old:"tf-ecs-instsg", New:"tf-ecs-inst-sg", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "ingress.#":*terraform.ResourceAttrDiff{Old:"2", New:"2", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.cidr_blocks.0":*terraform.ResourceAttrDiff{Old:"0.0.0.0/0", New:"0.0.0.0/0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.to_port":*terraform.ResourceAttrDiff{Old:"22", New:"22", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.protocol":*terraform.ResourceAttrDiff{Old:"-1", New:"-1", NewComputed:false, NewRemoved:false, NewExtra:"-1", RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.from_port":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "description":*terraform.ResourceAttrDiff{Old:"controls direct access to application instances", New:"controls direct access to application instances", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.security_groups.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.to_port":*terraform.ResourceAttrDiff{Old:"8080", New:"8080", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.protocol":*terraform.ResourceAttrDiff{Old:"tcp", New:"tcp", NewComputed:false, NewRemoved:false, NewExtra:"tcp", RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.from_port":*terraform.ResourceAttrDiff{Old:"8080", New:"8080", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyTainted:false}
    Diff Two (usually from apply): *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"ingress.864464978.protocol":*terraform.ResourceAttrDiff{Old:"tcp", New:"tcp", NewComputed:false, NewRemoved:false, NewExtra:"tcp", RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "name":*terraform.ResourceAttrDiff{Old:"tf-ecs-instsg", New:"tf-ecs-inst-sg", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "ingress.2701185744.security_groups.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.#":*terraform.ResourceAttrDiff{Old:"2", New:"2", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.to_port":*terraform.ResourceAttrDiff{Old:"8080", New:"8080", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.from_port":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.security_groups.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.from_port":*terraform.ResourceAttrDiff{Old:"22", New:"22", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "vpc_id":*terraform.ResourceAttrDiff{Old:"vpc-2c13de4b", New:"vpc-2c13de4b", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "owner_id":*terraform.ResourceAttrDiff{Old:"498302088937", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.protocol":*terraform.ResourceAttrDiff{Old:"-1", New:"-1", NewComputed:false, NewRemoved:false, NewExtra:"-1", RequiresNew:false, Sensitive:false, Type:0x0}, "egress.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.security_groups.3156370434":*terraform.ResourceAttrDiff{Old:"sg-3586a94c", New:"sg-3586a94c", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.cidr_blocks.0":*terraform.ResourceAttrDiff{Old:"0.0.0.0/0", New:"0.0.0.0/0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "description":*terraform.ResourceAttrDiff{Old:"controls direct access to application instances", New:"controls direct access to application instances", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.protocol":*terraform.ResourceAttrDiff{Old:"tcp", New:"tcp", NewComputed:false, NewRemoved:false, NewExtra:"tcp", RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.prefix_list_ids.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.cidr_blocks.#":*terraform.ResourceAttrDiff{Old:"1", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.cidr_blocks.0":*terraform.ResourceAttrDiff{Old:"43.230.88.117/32", New:"43.230.88.117/32", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.self":*terraform.ResourceAttrDiff{Old:"false", New:"false", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.864464978.from_port":*terraform.ResourceAttrDiff{Old:"8080", New:"8080", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "ingress.2701185744.to_port":*terraform.ResourceAttrDiff{Old:"22", New:"22", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.to_port":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "egress.482069346.security_groups.#":*terraform.ResourceAttrDiff{Old:"0", New:"0", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:true, DestroyTainted:false}

Also include as much context as you can about your config, state, and the steps you performed to trigger this error.

[mitchellh]

This looks like you opened the same issue twice? #10371

Closing as a dup, we'll get to it!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.