terraform-aws-modules / terraform-aws-vpc

Terraform module to create AWS VPC resources 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws
Apache License 2.0
2.99k stars 4.44k forks source link

China Terraform VPC endpoint issue : Unable to Create Endpoint( ec2, sqs, lambda, ecr.api, ecr.dkr ) #1121

Closed bnisarg1 closed 1 month ago

bnisarg1 commented 1 month ago

Description

while trying to create VPC endpoint in china using terraform-aws-modules/vpc/aws//modules/vpc-endpoints but getting error as reading EC2 VPC Endpoint Services: couldn't find resource

Note: for us-east-1, eu-west-1 we are able to create but for cn-northwest-1 and cn-north-1 we are unable to create

Below is the Error details

 Error: reading EC2 VPC Endpoint Services: couldn't find resource
│
│   ....data.aws_vpc_endpoint_service.this["lambda"],
│
│
╵
╷
│ Error: reading EC2 VPC Endpoint Services: couldn't find resource
│
│   with ....data.aws_vpc_endpoint_service.this["ecr.api"],

│ Error: reading EC2 VPC Endpoint Services: couldn't find resource
│
│   with ....data.aws_vpc_endpoint_service.this["ec2"],
│ 

If your request is for a new feature, please use the Feature request template.

Versions

Reproduction Code [Required]

module "vpc_interface_endpoints" {
  source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"

  endpoints = {
    for service in var.vpc_endpoints :
    service => {
      service             =  service,
      service_type        = "Interface",
      .....
      .....
  }
}
variable "vpc_endpoints" {
  description = "VPC Endpoints"
  type        = list(string)
  default     = ["ec2", "sns", "sqs", "ecr.api", "ecr.dkr", "lambda"]

}

Steps to reproduce the behavior:

1) terraform plan

Expected behavior

terraform plan should not return error and resources should be created once terraform apply is executed

Actual behavior

Getting following error for ec2, lambda, sqs, ecr.dkr, ecr.api

reading EC2 VPC Endpoint Services: couldn't find resource*

Terminal Output Screenshot(s)

EC2 Endpoint issue

Additional context

Note: for s3 and DynomoDB we didn't face issue

on evaluation we found that s3 and dynomodb are working because s3 and dynomodb supports service name as
1) com.amazonaws.cn-northwest-1.s3 2) cn.com.amazonaws.cn-northwest-1.s3

Below for both service name we are getting response image

image

image

But for ec2,sqs,ecr.dkr service name with com.amazonaws.cn-northwest-1.ec2 not supported

but with cn.com.amazonaws.cn-northwest-1.ec2 we get response as shown below image

bnisarg1 commented 1 month ago

Ok got it:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint_service

Specifying service will not work for non-AWS services or AWS services that don't follow the standard service_name pattern of com.amazonaws.<region>.<service>

service : ["ec2", "sns", "sqs", "ecr.api", "ecr.dkr", "lambda"]

But for china execpt sns other have cn.com which doesn't match with pattern

com.amazonawsaws.<region>.<service>

so we have to use serviceName instead of service

service Name:

ec2={service_name="cn.com.amazonaws.cn-northwest-1.ec2",service="ec2"}
sns={service_name="com.amazonaws.cn-northwest-1.sns",service="sns"}
sqs={service_name="cn.com.amazonaws.cn-northwest-1.sqs",service="sqs"}
ecr_api={service_name="cn.com.amazonaws.cn-northwest-1.ecr.api",service="ecr.api"}
ecr_dkr={service_name="cn.com.amazonaws.cn-northwest-1.ecr.dkr",service="ecr.dkr"}
lambda={service_name="cn.com.amazonaws.cn-northwest-1.lambda",service="lambda"}
module "vpc_interface_endpoints" {
source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"

endpoints = {
  for service, config in var.vpc_endpoints :

  service => {
     serviceName = is_china_region ? config.serviceName: null,
     service = is_china_region? null : config.service,
     service_type = "Interface",
     .....
     .....
    }
}

Note: Here if service name exists then use serviceName and make service null that would fix it else it will throw error if we provide both

Additional Info:

aws ec2 describe-vpc-endpoint-services      #will list as below. in that we can observe the ServiceName
 {
            "ServiceName": "com.amazonaws.cn-northwest-1.s3",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Gateway"
                }
            ],

{
            "ServiceName": "com.amazonaws.cn-northwest-1.sns",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],

{
            "ServiceName": "cn.com.amazonaws.cn-northwest-1.sqs",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],

{
            "ServiceName": "cn.com.amazonaws.cn-northwest-1.ecr.api",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],
{
            "ServiceName": "cn.com.amazonaws.cn-northwest-1.lambda",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],
  {
            "ServiceName": "cn.com.amazonaws.cn-northwest-1.ec2",
            "ServiceId": "vpce-svc-someid",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],
github-actions[bot] commented 5 days ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.