Tags for AWS resources created implicitly by other resources II

Benvorth commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Like #9061 but with different resource (aws_ec2_transit_gateway_vpc_attachment):

I have a TGW in account 1:

resource "aws_ec2_transit_gateway" "the_tgw" {
  amazon_side_asn                 = "65501"
  auto_accept_shared_attachments  = "enable"
  default_route_table_association = "disable"
  default_route_table_propagation = "disable"
  description                     = "My TGW in account 1"
  dns_support                     = "enable"
  vpn_ecmp_support                = "enable"
  tags = tomap({
    Name = "TGW"
  })
}

And share it via RAM with account 2.

Now I create a aws_ec2_transit_gateway_vpc_attachment in account 2 (different terraform-run) for the shared TGW:

data "aws_ec2_transit_gateway" "the_tgw" {
  filter {
    name = "options.amazon-side-asn"
    values = ["65501"]
  }
}

# <snip>create a VPC in account 2 with subnets </snip>

resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_attachment_to_tgw" {

  transit_gateway_id = data.aws_ec2_transit_gateway.the_tgw.id
  vpc_id             = ... # my vpc.ID
  subnet_ids         = ... # my subnet.IDs
  tags =tomap({
      "Name" = "TGW attachment for VPC in account 2"
  })
}

Terraform CLI and Terraform AWS Provider Version

> terraform -v
Terraform v1.0.3
on windows_amd64
+ provider registry.terraform.io/hashicorp/aws v3.59.0

Affected Resource(s)

aws_ec2_transit_gateway_vpc_attachment,

Expected Behavior

Within account 1 the TGW attachment's tags are visible

Actual Behavior

Attachment's tags not visible in account 1 (empty), only the tgw attachment itself.

References

9061

ewbankkit commented 3 years ago

@Benvorth Thanks for raising this issue.

Have you tried using the aws_ec2_tag resource in the 2nd account, specifying aws_ec2_transit_gateway_vpc_attachment.vpc_attachment_to_tgw.id as the resource_id?

Benvorth commented 3 years ago

Thanks for the hint.

I tried this:

resource "aws_ec2_tag" "vpc_attachment_to_TGW_Name_Tag" {
  resource_id = aws_ec2_transit_gateway_vpc_attachment.vpc_attachment_to_tgw.id
  key         = "Name"
  value       = "TGW attachment for VPC in account 2"
}

Unfortunately the attachments' name in account 1 is still empty...

yzguy commented 3 years ago

When you tag public or shared resources, the tags you assign are available only to your AWS account; no other AWS account will have access to those tags. For tag-based access control to shared resources, each AWS account must assign its own set of tags to control access to the resource.

Per https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html

The tags on a resource are separate per account. You would need to apply the tags in both accounts for it to work.

Benvorth commented 3 years ago

The tags on a resource are separate per account. You would need to apply the tags in both accounts for it to work.

Thanks for the link. Unfortunately AWS seems to use the "name" tag as an "name" identifier for the TGW attachment. That's why I am 80% sure that there is at least one scenario where it makes no sense to have the tags account-specific (the AWS recommended multi-accout strategy, the scenario I describe above):

Create a TGW in account 1,
In acccount 2 fetch TGW by APN
And add an VPC attachment to it in account 2

--> Now I could add tags in account 2 to this attachment (which works).

Following the provided "UserGuide" I have to add these tags (in my case name) to the TGW attachment in account 1, too.

But how would I fetch this resource in account 1 via Terraform? The TGW attachment created in account 2 doesn't have an APN, or a name-tag (yet). Only option would be via identifier or VPC-id - which both are not available in account 1 without manually copy/pasting values between the terraform scripts for account 1 and 2. Such a manual step would break my automatization pipeline.

Any other ideas or is this "just impossible with terraform" up to now and more of a shortcoming in AWS (and worth rising an issue there)?

yzguy commented 3 years ago

This is something I am currently facing as well (which is how I found this issue). Couple ways I'm considering below. It involves using local_file resource to pass around data, which isn't "great", because local_file resources will show changes in the plan any time the path changes, which will happen on different machines that run it (eg. my path to the TF code would be different than yours)

Basically the process is:

Account 1 shares the TGW to Account 2 and creates a tgw_data.yaml in the module where Account 2 is accepting and creating the TGW attachment
Account 2 reads the tgw_data.yaml to get values for the TGW RAM ARN, etc. and feeds them into the necessary resources
Account 2 then either creates a local file with it's account name as the filename, or it adds to a shared data file in Account 1's module where the TGW is shared
Account 1 then has some logic to load all the local files for each account or the shared data file to get data about their TGW attachments.

local.tgw_attachment_data will build a structure like this based on the file contents, it dynamically changes depending on what data is written into the file(s) from Account 2.

tgw_attachment_data = {
  "account2" = {
    "tgw_attachment_id" = "tgw-att-123456"
    "tgw_attachment_name" = "account2-name"
  }
  "account3" = {
    "tgw_attachment_id" = "tgw-att-67890"
    "tgw_attachment_name" = "account3-name"
  }
}

Account 1 uses this data to then loop through and tag all the attachments accordingly
- Tags will only be created on TGW attachments that are defined in the data file(s)

If we use that data structure with for_each on a local_file to demonstrate, you will only see the 2 created for which I had files for. This would be the same behavior if you used the same for_each on a aws_ec2_tag resource.

Terraform will perform the following actions:

  # local_file.file["account2"] will be created
  + resource "local_file" "file" {
      + content              = "TGW attachment for account2"
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "./account1_tgw-att-12345"
      + id                   = (known after apply)
    }

  # local_file.file["account3"] will be created
  + resource "local_file" "file" {
      + content              = "TGW attachment for account3"
      + directory_permission = "0755"
      + file_permission      = "0644"
      + filename             = "./account3_tgw-att-67890"
      + id                   = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Multiple Data Files

Account 1

locals {
  accounts_to_share_to = [
    "abc123",
    "123abc",
    "123456"
  ]

  # Build up data for TGW
  tgw_data = {
    tgw_id = data.aws_ec2_transit_gateway.tgw.id,
    tgw_name = local.tgw_name,
    tgw_ram_arn = data.aws_ram_resource_share.tgw_ram_rsh.arn,
    tgw_owner_id = data.aws_caller_identity.current.account_id
  }

  # Load all data files
  tgw_attachment_data = {
    for account in fileset(path.module, "*.yaml") :
     split(".", account)[0] => yamldecode(file(account))
  }
}

# Add accounts to resource share
resource "aws_ram_principal_association" "tgw" {
  for_each = local.accounts_to_share_to

  principal          = each.value
  resource_share_arn = data.aws_ram_resource_share.tgw_ram_rsh.arn
}

# Create a data file in each account with TGW information
resource "local_file" "tgw_data" {
  for_each = local.accounts_to_share_to

  content = yamlencode(local.tgw_data)
  filename = "${path.module}/../../${each.value}/us-east-1/tgw_accept/tgw_data.yaml"
  directory_permission = "0755"
  file_permission = "0644"
}

# Loop through account TGW attachment information and tag
resource "aws_ec2_tag" "vpc_attachment_tag" {
  for_each = local.tgw_attachment_data

  resource_id = each.value.tgw_attachment_id
  key         = "Name"
  value       = "TGW attachment for VPC in ${each.key}"
}

Account 2

locals {
   # Read data file from account 1 for TGW information
   tgw_data = yamldecode(file("${path.module}/tgw_data.yaml"))
   # Build up TGW attachment data
   tgw_attachment_data = {
     tgw_attachment_id = data.aws_ec2_transit_gateway_vpc_attachment.tgw_attach.id,
     tgw_attachment_vpc = data.aws_vpc.vpc.id
   }
}

data "aws_region" "current" {}

# Create VPC attachment using data from data file
resource "aws_ec2_transit_gateway_vpc_attachment" "tgw_attach" {
  subnet_ids = data.aws_subnet_ids.subnets.ids
  transit_gateway_id = local.tgw_data.tgw_id
  ...
}

# Create data file in TGW owner account with VPC attachment information
resource "local_file" "tgw_attach_data" {
  content = yamlencode(local.tgw_attachment_data)
  filename = "${path.module}/../../${local.tgw_data.tgw_owner_id}/${data.aws_region.current.name}/tgw_share/${data.aws_caller_identity.current.account_id}.yaml"
  directory_permission = "0755"
  file_permission = "0644"
}

Single Data File

Instead of making files per account you could use a single data file, and just have the account 2 see if it exists, if so then merge in it's data. The account 1 would read it if it exists, then tag anything defined in it appropriately

Account 2

locals {
  # Load shared data file if it exists
  data_file = fileexists("path/to/data_file.yaml") ? yamldecode(file("path/to/data_file.yaml")) : {}

  # Build up VPC attachment information
  tgw_attachment_data = {
    "${data.aws_caller_identity.current.account_id}" = {
      tgw_attachment_id = data.aws_ec2_transit_gateway_vpc_attachment.tgw_attach.id,
      tgw_attachment_vpc  = data.aws_vpc.vpc.id
    }
}

# Write VPC attachment information to shared data file
resource "local_file" "tgw_attachment_data" {
  content = yamlencode(merge(local.data_file, local.tgw_attachment_data))
  filename = "${path.module}/tgw_attachment_data.yaml"
  directory_permission = "0755"
  file_permission = "0644"
}

Account 1

locals {
  # Load shared data file if it exists
  data_file = fileexists("path/to/data_file.yaml") ? yamldecode(file("path/to/data_file.yaml")) : {}
}

# Loop through shared data and tag resources appropriately
resource "aws_ec2_tag" "vpc_attachment_tag" {
  for_each = local.data_file

  resource_id = each.value.tgw_attachment_id
  key         = "Name"
  value       = "TGW attachment for VPC ${each.value.tgw_attachment_vpc} in Account ${each.key}"
}

Both of these actually work pretty well, I've now tested both. The single file has more churn on it being written as in any account you share the TGW it's going to rewrite the file, then subsequently any other account already in the file will see it as changed, so it will write it again. I've opted for the multiple files as there would only be changed on the files that got run, and git won't care as long as the contents hasn't actually changed.

github-actions[bot] commented 1 year ago

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

github-actions[bot] commented 11 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws