search

terraform-aws-modules / terraform-aws-vpc

Terraform module to create AWS VPC resources 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws
Apache License 2.0
2.98k stars 4.43k forks source link

multiple VPC Endpoint Services matched if more than one endpoint of the same type on an account #686

Open cryptk opened 3 years ago

cryptk commented 3 years ago

Description

If creating multiple VPCs on the same account, and creating the same endpoint type in those VPCs, you receive the following error:

Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

Versions

Reproduction

Steps to reproduce the behavior: Create multiple VPCs using the terraform-aws-modules/vpc module create endpoints using the vpc-endpoints submodule, ensure that you create the same endpoint type in both VPCs

Expected behavior

Terraform should filter enough to differentiate endpoints that belong to different VPCs. Likely adding another filter for the VPC ID would fix this

Actual behavior

│ Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service
│ 
│   with module.lab[0].module.vpc_endpoints.data.aws_vpc_endpoint_service.this["appmesh_envoy_management"],
│   on .terraform/modules/lab.vpc_endpoints/modules/vpc-endpoints/main.tf line 9, in data "aws_vpc_endpoint_service" "this":
│    9: data "aws_vpc_endpoint_service" "this" {
bryantbiggs commented 3 years ago

hmm, looks like best we could do would be to add a conditional filter for tag https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeVpcEndpointServices.html

dirk39 commented 2 years ago

Hello everybody, I tried to replicate the issue using the module examples unsuccessfully. What I've noticed is that the problem comes with the custom service endpoints and not with the AWS-managed ones. Trying to add a filter per tag break the AWS service endpoints: 

Error: no matching EC2 VPC Endpoint Service found
   with module.vpc_endpoints_2.data.aws_vpc_endpoint_service.this["s3"],
   on terraform-aws-vpc/modules/vpc-endpoints/main.tf line 9, in data "aws_vpc_endpoint_service" "this":
    9: data "aws_vpc_endpoint_service" "this" {

One way to solve the issue could be by introducing a variable that determines if the service belongs to AWS or not. Another way (maybe easier) is just to define different service names.

What do you think?

Slancerk commented 1 year ago

use tags Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

rahulpatilgit commented 1 year ago

No issue was observed while testing. Able to create endpoints in two VPCs in same account.

module "vpc1" {
  source = "./modules/terraform-aws-vpc"
  name   = "vpc1"
}

module "endpoints1" {
  source             = "./modules/terraform-aws-vpc/modules/vpc-endpoints"
  vpc_id             = module.vpc1.vpc_id
  security_group_ids = [module.vpc1.default_security_group_id]
 endpoints = {
    s3 = {
      service = "s3"
      tags    = { Name = "s3-vpc-endpoint-vpc1" }
    },
    dynamodb = {
      service         = "dynamodb"
      service_type    = "Gateway"
      route_table_ids = flatten([module.vpc1.intra_route_table_ids, module.vpc1.private_route_table_ids, module.vpc1.public_route_table_ids])
      policy          = data.aws_iam_policy_document.dynamodb_endpoint_policy1.json
      tags            = { Name = "dynamodb-vpc-endpoint" }
    },
    ssm = {
      service             = "ssm"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      security_group_ids  = [module.vpc1.default_security_group_id]
    },
    ssmmessages = {
      service             = "ssmmessages"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      security_group_ids  = [module.vpc1.default_security_group_id]
    },
    lambda = {
      service             = "lambda"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
    },
    ecs = {
      service             = "ecs"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
    },
    ecs_telemetry = {
      create              = false
      service             = "ecs-telemetry"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
    },
    ec2 = {
      service             = "ec2"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      security_group_ids  = [module.vpc1.default_security_group_id]
    },
    ec2messages = {
      service             = "ec2messages"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      security_group_ids  = [module.vpc1.default_security_group_id]
    },
    ecr_api = {
      service             = "ecr.api"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      policy              = data.aws_iam_policy_document.generic_endpoint_policy1.json
    },
    ecr_dkr = {
      service             = "ecr.dkr"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      policy              = data.aws_iam_policy_document.generic_endpoint_policy1.json
    },
    kms = {
      service             = "kms"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
      security_group_ids  = [module.vpc1.default_security_group_id]
    },
    codedeploy = {
      service             = "codedeploy"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
    },
    codedeploy_commands_secure = {
      service             = "codedeploy-commands-secure"
      private_dns_enabled = true
      subnet_ids          = module.vpc1.private_subnets
    },
  }
}

data "aws_iam_policy_document" "generic_endpoint_policy1" {
  statement {
    effect    = "Deny"
    actions   = ["*"]
    resources = ["*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "StringNotEquals"
      variable = "aws:SourceVpc"

      values = [module.vpc1.vpc_id]
    }
  }
}

data "aws_iam_policy_document" "dynamodb_endpoint_policy1" {
  statement {
    effect    = "Deny"
    actions   = ["dynamodb:*"]
    resources = ["*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "StringNotEquals"
      variable = "aws:sourceVpce"

      values = [module.vpc1.vpc_id]
    }
  }
}

# =====================================================================================================

module "vpc2" {
  source = "./modules/terraform-aws-vpc"
  name   = "vpc2"
}

module "endpoints2" {
  source             = "./modules/terraform-aws-vpc/modules/vpc-endpoints"
  vpc_id             = module.vpc2.vpc_id
  security_group_ids = [module.vpc2.default_security_group_id]
 endpoints = {
    s3 = {
      service = "s3"
      tags    = { Name = "s3-vpc-endpoint-vpc2" }
    },
    dynamodb = {
      service         = "dynamodb"
      service_type    = "Gateway"
      route_table_ids = flatten([module.vpc2.intra_route_table_ids, module.vpc2.private_route_table_ids, module.vpc2.public_route_table_ids])
      policy          = data.aws_iam_policy_document.dynamodb_endpoint_policy2.json
      tags            = { Name = "dynamodb-vpc-endpoint" }
    },
    ssm = {
      service             = "ssm"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      security_group_ids  = [module.vpc2.default_security_group_id]
    },
    ssmmessages = {
      service             = "ssmmessages"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      security_group_ids  = [module.vpc2.default_security_group_id]
    },
    lambda = {
      service             = "lambda"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
    },
    ecs = {
      service             = "ecs"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
    },
    ecs_telemetry = {
      create              = false
      service             = "ecs-telemetry"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
    },
    ec2 = {
      service             = "ec2"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      security_group_ids  = [module.vpc2.default_security_group_id]
    },
    ec2messages = {
      service             = "ec2messages"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      security_group_ids  = [module.vpc2.default_security_group_id]
    },
    ecr_api = {
      service             = "ecr.api"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      policy              = data.aws_iam_policy_document.generic_endpoint_policy2.json
    },
    ecr_dkr = {
      service             = "ecr.dkr"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      policy              = data.aws_iam_policy_document.generic_endpoint_policy2.json
    },
    kms = {
      service             = "kms"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
      security_group_ids  = [module.vpc2.default_security_group_id]
    },
    codedeploy = {
      service             = "codedeploy"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
    },
    codedeploy_commands_secure = {
      service             = "codedeploy-commands-secure"
      private_dns_enabled = true
      subnet_ids          = module.vpc2.private_subnets
    },
  }
}

data "aws_iam_policy_document" "generic_endpoint_policy2" {
  statement {
    effect    = "Deny"
    actions   = ["*"]
    resources = ["*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "StringNotEquals"
      variable = "aws:SourceVpc"

      values = [module.vpc2.vpc_id]
    }
  }
}

data "aws_iam_policy_document" "dynamodb_endpoint_policy2" {
  statement {
    effect    = "Deny"
    actions   = ["dynamodb:*"]
    resources = ["*"]

    principals {
      type        = "*"
      identifiers = ["*"]
    }

    condition {
      test     = "StringNotEquals"
      variable = "aws:sourceVpce"

      values = [module.vpc2.vpc_id]
    }
  }
}