aws_ec2_transit_gateway_vpc_attachment does not set transit_gateway_default_route_table_association or transit_gateway_default_route_table_propagation correctly

BrandonsAccount commented 5 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Issue reproduced on 11.11 and 11.13.

I can confirm that this was not a bug in aws provider version 2.0.0. The issue exists in version 2.7.0.

Affected Resource(s)

aws_ec2_transit_gateway_vpc_attachment

Terraform Configuration Files

resource "aws_ec2_transit_gateway_vpc_attachment" "this" {
  subnet_ids                                      = ["${aws_subnet.private.*.id}"]
  transit_gateway_id                              = "${var.transit_gateway_id}"
  vpc_id                                          = "${aws_vpc.this.id}"
  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false
  tags = "${merge(var.tags, map("Name", format("%s-attachment", var.name)))}"
}

This is the plan output

  + aws_ec2_transit_gateway_vpc_attachment.this
      id:                                              <computed>
      dns_support:                                     "enable"
      ipv6_support:                                    "disable"
      subnet_ids.#:                                    "4"
      subnet_ids.104XXXXXX:                           "subnet-02b3XXXXXXXXXXX"
      subnet_ids.178XXXXXX:                           "subnet-0cb3XXXXXXXXXXX"
      subnet_ids.207XXXXXX:                           "subnet-0daXXXXXXXXXXX"
      subnet_ids.264148865:                            "subnet-001XXXXXXXXXXXX"
      tags.%:                                          "1"
      tags.Name:                                       "vpc-attachment"
      transit_gateway_default_route_table_association: "false"
      transit_gateway_default_route_table_propagation: "false"
      transit_gateway_id:                              "tgw-XXXXXXXXXXXXXXXXXX"
      vpc_id:                                          "vpc-0c8XXXXXXXXXXXXXXXXXX"
      vpc_owner_id:                                    <computed>

Expected Behavior

Running terraform state show aws_ec2_transit_gateway_vpc_attachment.this should show this...

transit_gateway_default_route_table_association = false
transit_gateway_default_route_table_propagation = false

It also should not detect drift immediately after applying these changes.

Actual Behavior

Running terraform state show aws_ec2_transit_gateway_vpc_attachment.this shows this...

transit_gateway_default_route_table_association = true
transit_gateway_default_route_table_propagation = true

When running terraform apply a second time, the following drift is reported (but the apply runs successfully). It looks like this...

  ~ aws_ec2_transit_gateway_vpc_attachment.this
      transit_gateway_default_route_table_association: "true" => "false"
      transit_gateway_default_route_table_propagation: "true" => "false"

The terraform state shows the correct values after the 2nd apply...

transit_gateway_default_route_table_association = false
transit_gateway_default_route_table_propagation = false

Every terraform apply moving forward continues to detect drift though...

  ~ aws_ec2_transit_gateway_vpc_attachment.this
      transit_gateway_default_route_table_association: "true" => "false"
      transit_gateway_default_route_table_propagation: "true" => "false"

Steps to Reproduce

There's a fair amount of setup involved so I don't know that I can cover it all...

Create an organization within your aws account
Create an aws account for your transit gateway
Create a transit gateway with default route table association and propagation disabled
Create two route tables for the tgw. One to associate with and one to propagate to
Use RAM to share the tgw to all accounts in your org
Create an aws account and create a VPC in it
In this account use terraform to attach the VPC to the transit gateway that should be shared with it

Important Factoids

The transit gateway attachment creates successfully and associates successfully with the tgw. It even propagates successfully

furtber commented 5 years ago

I have basically a "me-too" here, but on 0.12-beta2

+ terraform --version -no-color
Terraform v0.12.0-beta2
+ provider.aws v2.7.0

and I'm sure it was the same with this combo

+ terraform --version -no-color
Terraform v0.12.0-beta1
+ provider.aws v2.6.0

jeffmccollum commented 5 years ago

We are having this issue as well. One thing to note, attachments for VPCs in the same account as the transit gateway do not exhibit this behavior. This only occurs with attachments that are created in another account using RAM.

ewbankkit commented 5 years ago

https://github.com/terraform-providers/terraform-provider-aws/issues/8600

ewbankkit commented 5 years ago

@BrandonsAccount @piersf @jeffmccollum @furtber The transit_gateway_default_route_table_association and transit_gateway_default_route_table_propagation attributes don't do anything for cross-account VPC attachments as hinted at in the documentation:

This cannot be configured or perform drift detection with Resource Access Manager shared EC2 Transit Gateways.

but I agree we should do something to try and stop this drift being detected in case the attributes are set mistakenly, especially since a second terraform apply for me gave an error:

aws_ec2_transit_gateway_vpc_attachment.test: Modifying... (ID: tgw-attach-0000000000000000)
  transit_gateway_default_route_table_association: "true" => "false"
  transit_gateway_default_route_table_propagation: "true" => "false"

Error: Error applying plan:

1 error(s) occurred:

* aws_ec2_transit_gateway_vpc_attachment.test: 1 error(s) occurred:

* aws_ec2_transit_gateway_vpc_attachment.test: error updating EC2 Transit Gateway Attachment (tgw-attach-0000000000000000) Route Table (tgw-rtb-0000000000000000) association: error determining EC2 Transit Gateway Attachment Route Table (tgw-rtb-0000000000000000) association (tgw-attach-0000000000000000): InvalidRouteTableID.NotFound: Transit Gateway Route Table tgw-rtb-0000000000000000 was deleted or does not exist.
    status code: 400, request id: 2993f92d-230f-46f0-a595-21729fe95789

For cross-account VPC attachments maybe the correct resource to specify the default route table attributes on is the vpc_attachment_accepter resource? See https://github.com/terraform-providers/terraform-provider-aws/pull/8679#issuecomment-493797516.

michelzanini commented 4 years ago

This is my Transit Gateway:

module "transit_gateway" {
  source  = "terraform-aws-modules/transit-gateway/aws"
  version = "~> 1.0"
  name    = "transit-gateway"

  enable_auto_accept_shared_attachments  = true
  enable_default_route_table_association = false
  enable_default_route_table_propagation = false

  ram_principals = var.ram_principals
}

As you see enable_default_route_table_association and enable_default_route_table_propagation are false as that's what I need.

Then my VPC attachment initially was this:

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_attachment" {
  transit_gateway_id = var.transit_gateway_id
  vpc_id             = var.vpc_id
  subnet_ids         = var.subnet_ids

  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false
}

But, by using the above I run into this issue. It works ok on the same account as the transit gateway, but it ALWAYS detect drift on state on accounts that I shared the gateway with:

transit_gateway_default_route_table_association: "true" => "false"
transit_gateway_default_route_table_propagation: "true" => "false"

To "escape" from this drift issue I set it to true. Which then "resolves" the issue for the accounts which I have shared the TGW with. But, then when running this same code (which is a shared module for me) on the same account as the TGW it fails with the error found on issue https://github.com/terraform-providers/terraform-provider-aws/issues/13512.

So in the end, either I have the issue here https://github.com/terraform-providers/terraform-provider-aws/issues/8383 when using false or I have the issue there https://github.com/terraform-providers/terraform-provider-aws/issues/13512 when using true.

So in the end what I have done was this:

data "aws_caller_identity" "current" {}

data "aws_ec2_transit_gateway" "transit_gateway" {
  id = var.transit_gateway_id
}

locals {
  aws_account_id           = data.aws_caller_identity.current.account_id
  transit_gateway_owner_id = data.aws_ec2_transit_gateway.transit_gateway.owner_id

  different_account_as_transit_gateway = local.aws_account_id != local.transit_gateway_owner_id
}

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_attachment" {
  for_each = var.vpc_subnets_to_attach

  transit_gateway_id = var.transit_gateway_id
  vpc_id             = var.vpc_id
  subnet_ids         = var.subnet_ids

  transit_gateway_default_route_table_association = local.different_account_as_transit_gateway
  transit_gateway_default_route_table_propagation = local.different_account_as_transit_gateway
}

This detects if the account is the same or not as the TGW and sets the values to "escape" both issues.

But what I really need is this drift issue to be fixed so I can come back to set transit_gateway_default_route_table_association and transit_gateway_default_route_table_propagation to false as they should be in my use case.

michelzanini commented 4 years ago

Any chance we can fix this annoying problem?

michelzanini commented 4 years ago

Another solution as workaround:

  transit_gateway_default_route_table_association = false
  transit_gateway_default_route_table_propagation = false

  lifecycle {
    ignore_changes = [
      transit_gateway_default_route_table_association,
      transit_gateway_default_route_table_propagation
    ]
  }

piersf commented 2 years ago

Has there been a fix for this? The issue is still present in Terraform 1.1.4 and AWS Provider 3.70.0

The same issue is mentioned in https://github.com/hashicorp/terraform-provider-aws/issues/8600

Thank you!

AdolfoBolivarO commented 2 years ago

issue is still present, i am using Terraform v1.1.9 and aws provider v4.17.1

duclm2609 commented 2 years ago

The issue is really annoying. It happens on cross-account VPC attachment. Using the workaround by @michelzanini while waiting for the official fix.

ewbankkit commented 2 years ago

Relates https://github.com/hashicorp/terraform-provider-aws/issues/8383.

galiceau commented 2 years ago

me too. I have the same issue. This issue exists from 3 years. No fix until now :( I see one workaround : managing another table instead of default table. In this way we have to manage all propagations and associations "manually". Additionally, if I set for a cross-account : transit_gateway_default_route_table_association = false transit_gateway_default_route_table_propagation = false the executed plan sets the values to true.... big bug... Terraform v1.2.5 on linux_amd64, provider registry.terraform.io/hashicorp/aws v4.23.0

gdavison commented 6 months ago

Based on testing, this appears to have been resolved. If you're still encountering this issue, please open a new issue.

github-actions[bot] commented 6 months ago

[!WARNING] This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

github-actions[bot] commented 5 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws