datasource - aws_network_interface error

chenshap commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.6 aws v4.12.1

Affected Resource(s)

datasource: aws_network_interface aws_lb_target_group_attachment

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_vpc_endpoint" "execute_api" {
  vpc_id             = var.vpc_id
  service_name       = "com.amazonaws.us-east-1.execute-api"
  vpc_endpoint_type  = "Interface"
  subnet_ids         = var.eks_one_subnet_per_az_private_list
  security_group_ids = [module.https-self.security_group_id]

  tags = {
    Name = "${var.project_name}-${var.environment}-${var.api_name}-rest-api"
  }
}

data "aws_network_interface" "apigw_endpoint_eni" {
  for_each = aws_vpc_endpoint.execute_api.network_interface_ids

  id = each.value

  depends_on = [aws_vpc_endpoint.execute_api]
}

resource "aws_lb_target_group_attachment" "rest_api_lb_ga" {
  for_each = data.aws_network_interface.apigw_endpoint_eni

  target_group_arn = aws_lb_target_group.rest_api_lb_tg.arn
  target_id        = each.value.private_ip
  port             = 443
}

Expected Behavior

get information about a Network Interfaces

Actual Behavior

got the error :


 Error: Invalid for_each argument

  on ../../../modules/rest-api/custom_domain_name.tf line 54, in data "aws_network_interface" "apigw_endpoint_eni":
   54:   for_each = aws_vpc_endpoint.execute_api.network_interface_ids
    ├────────────────
    │ aws_vpc_endpoint.execute_api.network_interface_ids is a set of string, known only after apply

 The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many
instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends
 on.

```hcl
### Steps to Reproduce

1. `terraform apply`

justinretzolk commented 2 years ago

Hey @chenshap 👋 Thank you for taking the time to raise this! This error is caused by a limitation of the for_each meta argument of Terraform, where the value passed to for_each must be known values. The for_each meta-argument documentation covers this with the following excerpt:

The keys of the map (or all the values in the case of a set of strings) must be known values, or you will get an error message that for_each has dependencies that cannot be determined before apply, and a -target may be needed.

To get around this, you'll need to either do a targeted apply first, use a known value, or break these configurations out such that the aws_vpc_endpoint resource(s) are in a separate configuration that has an output value that contains the network_interface_ids values, which the separate configuration containing the for_each can consume.

gaperSmile commented 1 year ago

I have the same problem. With terrafom 0.15.5 and aws = { source = "hashicorp/aws" version = "4.14.0" }

gaperSmile commented 1 year ago

I have found a workaround using the network_interfaces data to filtering by security group like bellow:

resource "aws_security_group" "my_endpoint_group" {
  vpc_id      = var.vpc_id

  ingress {
    protocol    = -1
    description = "example"
    cidr_blocks = ["0.0.0.0/0"]
    from_port   = 0
    to_port     = 0
  }

  egress {
    protocol    = -1
    description = "example"
    cidr_blocks = ["0.0.0.0/0"]
    from_port   = 0
    to_port     = 0
  }
}

resource "aws_vpc_endpoint" "iot_core" {
  vpc_id            = var.vpc_id
  service_name      = "com.amazonaws.eu-west-1.iot.data"
  vpc_endpoint_type = "Interface"

  security_group_ids = [
    aws_security_group.my_endpoint_group.id
  ]
  subnet_ids = var.private_subnets_ids
}

data "aws_network_interfaces" "my_network_intefaces" {
  filter {
    name   = "group-id"
    values = [aws_security_group.my_endpoint_group.id]
  }
  depends_on = [aws_vpc_endpoint.iot_core]
}

data "aws_network_interface" "iot_core_endpoint" {
  count = length(var.private_subnets_ids)
  id    = data.aws_network_interfaces.my_network_intefaces.ids[count.index]
}

It's a lot of work just for getting something already accessible as output of aws_vpc_endpoint... I think a fix is mandatory.

github-actions[bot] commented 5 days ago

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

hashicorp / terraform-provider-aws