terraform-aws-modules / terraform-aws-vpc

Terraform module to create AWS VPC resources πŸ‡ΊπŸ‡¦
https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws
Apache License 2.0
2.92k stars 4.36k forks source link

IPv6 Route in Route Table (rtb-xxx) with destination (::/0) already exists #1058

Closed wiseelf closed 3 months ago

wiseelf commented 3 months ago

Description

During the creation of a new VPC with IPv6 support, I encountered the "RouteAlreadyExists" issue for private subnets. Specifically, there are six private subnets spread across three availability zones (AZs).

For IPv4, routes are successfully created because their count is determined by the number of NAT gateways: count = local.create_vpc && var.enable_nat_gateway ? local.nat_gateway_count : 0. However, with IPv6, duplicate routes occur due to the count of routes linked to local.len_private_subnets: count = local.create_vpc && var.enable_nat_gateway ? local.nat_gateway_count : 0.

Consequently, even though there are only three route tables for each AZ when there are six private subnets across three AZs, Terraform attempts to create six routes to "::/0" for the EIGW (Egress Internet Gateway), resulting in the following error.

β”‚ Error: RouteAlreadyExists: Route in Route Table (rtb-xxx) with destination (::/0) already exists
β”‚
β”‚   with aws_route.private_ipv6_egress[4],
β”‚   on main.tf line 1023, in resource "aws_route" "private_ipv6_egress":
β”‚ 1023: resource "aws_route" "private_ipv6_egress" {
β”‚
β•΅

Versions

Reproduction Code [Required]

Steps to reproduce the behavior:

create 6 private subnets, enable ipv6 and specify:

  private_subnets = [
    "10.0.0.0/24", "10.0.1.0/24", "10.0.2.0/24",
    "10.0.3.0/24", "10.0.4.0/24", "10.0.5.0/24",
  ]

  private_subnet_ipv6_prefixes = [
    1, 2, 3,
    4, 5, 6
  ]
  enable_nat_gateway     = true
  single_nat_gateway     = false
  one_nat_gateway_per_az = true
  create_egress_only_igw = true

Expected behavior

No error Error: RouteAlreadyExists: Route in Route Table (rtb-xxx) with destination (::/0) already exists

Actual behavior

Error: RouteAlreadyExists: Route in Route Table (rtb-xxx) with destination (::/0) already exists

RunCor399 commented 3 months ago

I'm actually having the same problem, although in my setup I have 3 public, 3 private and 3 database subnets spread in 3 AZs

RunCor399 commented 3 months ago

Ok I believe I've found the root cause of the problem. In my scenario I'm trying to deploy a VPC with 3 public, 3 private and 3 database subnets, furthermore I opted for having a single NAT gateway by setting "enable_nat_gateway = true" and "single_nat_gateway = true".

Performing some tests I've noticed that the terraform plan is trying to create 3 different "aws_route.private_ipv6_egress" routes, but the problem is that since only a single NAT Gateway is created (hence a single Route Table as you can notice from this comment the 3 created routes won't of course fit in the single Route Table.

This line is where supposedly each egress_ipv6 route should be associated with a different Route Table, but of course the count of the block will be equal to three while the aws_route_table.private[] will only have index 0.

To confirm my ipothesis I've tried to re-deploy the module using the same configuration but deploying one NAT Gateway per subnet ("enable_nat_gateway = true", "one_nat_gateway_per_az = false" and "single_nat_gateway = false") instead of a single one... and it worked.

Let me know if in your opinion I got something wrong

andrewleegoss1978 commented 3 months ago

I also just starting getting this error this week. I am testing possible workarounds. I suspect it is related to a recent Terraform code change.

RunCor399 commented 3 months ago

Ok I believe I've found the root cause of the problem. In my scenario I'm trying to deploy a VPC with 3 public, 3 private and 3 database subnets, furthermore I opted for having a single NAT gateway by setting "enable_nat_gateway = true" and "single_nat_gateway = true".

Performing some tests I've noticed that the terraform plan is trying to create 3 different "aws_route.private_ipv6_egress" routes, but the problem is that since only a single NAT Gateway is created (hence a single Route Table as you can notice from this comment the 3 created routes won't of course fit in the single Route Table.

This line is where supposedly each egress_ipv6 route should be associated with a different Route Table, but of course the count of the block will be equal to three while the aws_route_table.private[] will only have index 0.

To confirm my ipothesis I've tried to re-deploy the module using the same configuration but deploying one NAT Gateway per subnet ("enable_nat_gateway = true", "one_nat_gateway_per_az = false" and "single_nat_gateway = false") instead of a single one... and it worked.

Let me know if in your opinion I got something wrong

In my case I solved the problem by modifying the amount of ipv6 egress routes created (one per each NAT gateway). In this way I was able to deploy a single NAT Gateway for my 3 private subnets and associate to it a single ipv6 egress route.

To do so I've replaced local.len_private_subnets with local.nat_gateway_count Here

fideloper commented 3 months ago

I just hit this issue, and this saved me hours since I probably wouldn't have noticed I kept "single_nat_gateway = true" in there by accident πŸ˜…, thanks @RunCor399 !

antonbabenko commented 3 months ago

This issue has been resolved in version 5.7.1 :tada:

andrewleegoss1978 commented 3 months ago

Thank you @antonbabenko.

antonbabenko commented 3 months ago

@andrewleegoss1978 It is all @bryantbiggs who reviewed it and merged it :)

github-actions[bot] commented 1 month ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.