search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.74k stars 9.1k forks source link

Terraform detects changes made with terraform as outside changes #22317

Open zoltan-toth-mw opened 2 years ago

zoltan-toth-mw commented 2 years ago

Community Note

Terraform Version

Terraform v1.1.2
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.70.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Terraform Configuration Files

resource "aws_route" "kubernetes_destinations" {
  for_each = local.route_tables

  route_table_id = each.key
  destination_cidr_block = "10.103.0.0/16"
  transit_gateway_id = data.aws_ec2_transit_gateway.tgw.id
}

Expected Behavior

It should not report changes made with terraform as changes from the outside.

Actual Behavior

It reports changes made with terraform as they were made outside of it.

Steps to Reproduce

  1. I added the following resource to my terraform config:
resource "aws_route" "kubernetes_destinations" {
  for_each = local.route_tables

  route_table_id = each.key
  destination_cidr_block = "10.103.0.0/16"
  transit_gateway_id = data.aws_ec2_transit_gateway.tgw.id
}
  1. I ran terraform apply. It already complained about my previous apply like it was made outside of terraform.

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

module.vpc.aws_route_table.private[0] has changed

~ resource "aws_route_table" "private" { id = "rtb-0b1REDACTED" ~ route = [

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

Terraform will perform the following actions:

aws_route.kubernetes_destinations["rtb-0b14REDACTED"] will be created

Plan: 3 to add, 0 to change, 0 to destroy. aws_route.kubernetes_destinations["rtb-08eREDACTED"]: Creating... [...]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.


3. I ran terraform plan and I saw my recently apply changes reported as non terraform changes:
```terraform

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # module.vpc.aws_route_table.private[0] has changed
  ~ resource "aws_route_table" "private" {
        id               = "rtb-0b1REDACTED"
      ~ route            = [
          + {
              + carrier_gateway_id         = ""
              + cidr_block                 = "10.103.0.0/16"
              + destination_prefix_list_id = ""
              + egress_only_gateway_id     = ""
              + gateway_id                 = ""
              + instance_id                = ""
              + ipv6_cidr_block            = ""
              + local_gateway_id           = ""
              + nat_gateway_id             = ""
              + network_interface_id       = ""
              + transit_gateway_id         = "tgw-0e8REDACTED"
              + vpc_endpoint_id            = ""
              + vpc_peering_connection_id  = ""
            },
            # (2 unchanged elements hidden)
        ]
        tags             = {
            "Name" = "staging-private-a"
        }
        # (5 unchanged attributes hidden)
    }

  # module.vpc.aws_route_table.private[1] has changed
  #...
  1. Checked the remote state file, the changes are there though... 
    {
      "index_key": "rtb-0b1REDACTED",
      "schema_version": 0,
      "attributes": {
        "carrier_gateway_id": "",
        "destination_cidr_block": "10.103.0.0/16",
        "destination_ipv6_cidr_block": "",
        "destination_prefix_list_id": "",
        "egress_only_gateway_id": "",
        "gateway_id": "",
        "id": "r-rtb-0b14REDACTED",
        "instance_id": "",
        "instance_owner_id": "",
        "local_gateway_id": "",
        "nat_gateway_id": "",
        "network_interface_id": "",
        "origin": "CreateRoute",
        "route_table_id": "rtb-0b14REDACTED",
        "state": "active",
        "timeouts": null,
        "transit_gateway_id": "tgw-0e8REDACTED",
        "vpc_endpoint_id": "",
        "vpc_peering_connection_id": ""
      },
      "sensitive_attributes": [],
      "private": "REDACTED",
      "dependencies": [
        "data.aws_ec2_transit_gateway.tgw",
        "module.vpc.aws_route_table.private"
      ]
    }
  2. Note that the complaint that was there before the apply is not reported anymore, only the most recent changes, the latest apply is reported. I saw this issue after I created the vpc too.

Notes

Reported it as a generic terraform problem before: https://github.com/hashicorp/terraform/issues/30239

ewbankkit commented 2 years ago

@zoltan-toth-mw Thanks for raising this issue. Does your aws_route_table.private configuration contain any route{} blocks?

simonweil commented 2 years ago

This issue happens also when creating roles

zoltan-toth-mw commented 2 years ago

Hey @ewbankkit , no route{} blocks, I'm using it just like in the example I provided.

resource "aws_route" "kubernetes_destinations" {
  for_each = local.route_tables

  route_table_id = each.key
  destination_cidr_block = "10.103.0.0/16"
  transit_gateway_id = data.aws_ec2_transit_gateway.tgw.id
}
zoltan-toth-mw commented 2 years ago

Happens with other resources too:

Terraform detected the following changes made outside of Terraform since the
last "terraform apply":

  # aws_default_security_group.default has changed
  ~ resource "aws_default_security_group" "default" {
        id                     = "sg-0107REDACTED"
        name                   = "default"
      + tags                   = {}
        # (8 unchanged attributes hidden)
    }
ilyagorban-codefresh commented 2 years ago

Happens on aws_security_group when you have aws_security_group_rule as well

robinbowes commented 2 years ago

We're seeing something similar to this with 0.15.4.

We manage SGs and SG rules in separate Terraform Cloud workspaces. We create all groups centrally in one workspace, and manage the rules for each group in the service-related workspaces. We're seeing the central SG workspace detect any changes made to SG rules made in the service-related workspaces, and report them:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply":
yfauser commented 1 year ago

I tried to recreate the issue using routes pointing to transitgateway with 1.2.9 and the latest aws provider code from main, but I don't see the issue Note: Objects have changed outside of Terraform. @zoltan-toth-mw, are you still seeing the issue?

Michagogo commented 4 months ago

Hey @ewbankkit , no route{} blocks, I'm using it just like in the example I provided.

resource "aws_route" "kubernetes_destinations" {
  for_each = local.route_tables

  route_table_id = each.key
  destination_cidr_block = "10.103.0.0/16"
  transit_gateway_id = data.aws_ec2_transit_gateway.tgw.id
}

I think the question was not about the aws_route resource, but rather the aws_route_table resource which is the resource with detected changes. It’s documented behavior for many “subresources”that exist as separate resources in the provider but can also be configured as blocks in parent resources (e.g. route table routes, IAM identity policies, security group rules, …) that if the parent resource has configuration blocks present for those subresources, that parent resource instance assumes management of the set of whatever the applicable subresource is, and those shouldn’t also be created separately. If the applicable configuration is absent, it’s left unmanaged and in that case they can be created as their own separate resource instances. So if the route table instance has any route{} blocks defined, all routes for that route table should be defined within the route table resource. If for whatever reason that isn’t possible, then all routes defined inline need to be split out into separate route resources.