[Bug]: Timeout error when changing destination in route table from NAT to IGW

[alfcleverley]

Terraform Core Version

1.3.5

AWS Provider Version

4.44.0

Affected Resource(s)

• aws_route

Expected Behavior

No error when changing the destination in the route table from NAT to IGW. ex: Change the destination in the route table from IGW to NAT. →No error occurs.

$ terraform apply
aws_route.igw_route: Refreshing state... [id=r-rtb-0f6b38cfda7d0f7dd1080289494]
aws_vpc.test_vpc: Refreshing state... [id=vpc-025b58fadb7951954]
aws_eip.test_eip: Refreshing state... [id=eipalloc-0aae8735d95b7293e]
aws_subnet.test_subnet: Refreshing state... [id=subnet-0c80d439cd69915a8]
aws_route_table.test_route_table: Refreshing state... [id=rtb-0f6b38cfda7d0f7dd]
aws_internet_gateway.test_igw: Refreshing state... [id=igw-029f18f50b4bcf284]
aws_nat_gateway.test_nat: Refreshing state... [id=nat-0a37ccf75e56205a0]

Terraform used the selected providers to generate the following execution plan. Resource actions
are indicated with the following symbols:
  + create
  - destroy

Terraform will perform the following actions:

  # aws_route.igw_route will be destroyed
  # (because aws_route.igw_route is not in configuration)
  - resource "aws_route" "igw_route" {
      - destination_cidr_block = "0.0.0.0/0" -> null
      - gateway_id             = "igw-029f18f50b4bcf284" -> null
      - id                     = "r-rtb-0f6b38cfda7d0f7dd1080289494" -> null
      - origin                 = "CreateRoute" -> null
      - route_table_id         = "rtb-0f6b38cfda7d0f7dd" -> null
      - state                  = "active" -> null
    }

  # aws_route.nat_route will be created
  + resource "aws_route" "nat_route" {
      + destination_cidr_block = "0.0.0.0/0"
      + id                     = (known after apply)
      + instance_id            = (known after apply)
      + instance_owner_id      = (known after apply)
      + nat_gateway_id         = "nat-0a37ccf75e56205a0"
      + network_interface_id   = (known after apply)
      + origin                 = (known after apply)
      + route_table_id         = "rtb-0f6b38cfda7d0f7dd"
      + state                  = (known after apply)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_route.igw_route: Destroying... [id=r-rtb-0f6b38cfda7d0f7dd1080289494]
aws_route.nat_route: Creating...
aws_route.igw_route: Destruction complete after 0s
aws_route.nat_route: Creation complete after 1s [id=r-rtb-0f6b38cfda7d0f7dd1080289494]

Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

Actual Behavior

When changing the destination in the route table from NAT to IGW, a timeout error occurs.

$ terraform apply
aws_eip.test_eip: Refreshing state... [id=eipalloc-0aae8735d95b7293e]
aws_route.nat_route: Refreshing state... [id=r-rtb-0f6b38cfda7d0f7dd1080289494]
aws_vpc.test_vpc: Refreshing state... [id=vpc-025b58fadb7951954]
aws_internet_gateway.test_igw: Refreshing state... [id=igw-029f18f50b4bcf284]
aws_route_table.test_route_table: Refreshing state... [id=rtb-0f6b38cfda7d0f7dd]
aws_subnet.test_subnet: Refreshing state... [id=subnet-0c80d439cd69915a8]
aws_nat_gateway.test_nat: Refreshing state... [id=nat-0a37ccf75e56205a0]

Terraform used the selected providers to generate the following execution plan. Resource actions
are indicated with the following symbols:
  + create
  - destroy

Terraform will perform the following actions:

  # aws_route.igw_route will be created
  + resource "aws_route" "igw_route" {
      + destination_cidr_block = "0.0.0.0/0"
      + gateway_id             = "igw-029f18f50b4bcf284"
      + id                     = (known after apply)
      + instance_id            = (known after apply)
      + instance_owner_id      = (known after apply)
      + network_interface_id   = (known after apply)
      + origin                 = (known after apply)
      + route_table_id         = "rtb-0f6b38cfda7d0f7dd"
      + state                  = (known after apply)
    }

  # aws_route.nat_route will be destroyed
  # (because aws_route.nat_route is not in configuration)
  - resource "aws_route" "nat_route" {
      - destination_cidr_block = "0.0.0.0/0" -> null
      - id                     = "r-rtb-0f6b38cfda7d0f7dd1080289494" -> null
      - nat_gateway_id         = "nat-0a37ccf75e56205a0" -> null
      - origin                 = "CreateRoute" -> null
      - route_table_id         = "rtb-0f6b38cfda7d0f7dd" -> null
      - state                  = "active" -> null
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_route.nat_route: Destroying... [id=r-rtb-0f6b38cfda7d0f7dd1080289494]
aws_route.igw_route: Creating...
aws_route.igw_route: Creation complete after 0s [id=r-rtb-0f6b38cfda7d0f7dd1080289494]
aws_route.nat_route: Still destroying... [id=r-rtb-0f6b38cfda7d0f7dd1080289494, 10s elapsed]
　~(abbr)~
aws_route.nat_route: Still destroying... [id=r-rtb-0f6b38cfda7d0f7dd1080289494, 4m50s elapsed]
aws_route.nat_route: Still destroying... [id=r-rtb-0f6b38cfda7d0f7dd1080289494, 5m0s elapsed]
?
„  Error: error waiting for Route in Route Table (rtb-0f6b38cfda7d0f7dd) with destination (0.0.0.0/0) to delete: timeout while waiting for resource to be gone (last state: 'ready', timeout: 5m0s)

Relevant Error/Panic Output Snippet

* The event is reproduced by commenting out or uncommenting the "aws_route" block.
* Both "NAT to IGW" and "IGW to NAT" succeed or fail.(sometimes)
* The routing change is completed on the AWS console.
* It is not an error that the routing cannot be erased based on something.
　  →I have not been able to solve the problem by adding "lifecycle_create_before_destroy".
* The command execution to the AWS side is going well, but it seems that the return value is not obtained on the terraform side.

* Is it deprecated to change the route by deleting or increasing the entire "aws_route" block?
　→I am using "terraform code" as a module.
　→The code I have described is an excerpted version of the code to make the error part into one terraformfile.
　→Since the code is modularized, we would like to deal with it by deleting or increasing the entire "aws_route" block, rather than simply changing the parameters in the "aws_route" block.

Terraform Configuration Files

main.tf

resource aws_vpc test_vpc {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "test-rt-vpc"
  }
}

resource aws_subnet test_subnet {
  vpc_id            = aws_vpc.test_vpc.id
  cidr_block        = "10.0.0.0/24"
  availability_zone = "ap-northeast-1a"
  tags = {
    Name = "test-subnet-rt"
  }
}

resource aws_internet_gateway test_igw {
  vpc_id = aws_vpc.test_vpc.id
  tags = {
    Name = "test-igw-rt"
  }
}

resource aws_eip test_eip {
  vpc      = true
}

resource aws_nat_gateway test_nat {
  allocation_id = aws_eip.test_eip.id
  subnet_id     = aws_subnet.test_subnet.id
  tags = {
    Name = "test-NAT-rt"
  }
}

resource aws_route_table test_route_table {
  vpc_id = aws_vpc.test_vpc.id
  tags = {
    Name = "test-routetable-rt"
  }
}

#resource aws_route igw_route {
#  route_table_id         = aws_route_table.test_route_table.id
#  destination_cidr_block = "0.0.0.0/0"
#  gateway_id             = aws_internet_gateway.test_igw.id
#}

resource aws_route nat_route {
  route_table_id         = aws_route_table.test_route_table.id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.test_nat.id
}

→The event is reproduced by commenting out or uncommenting the "aws_route" block.

version.tf

terraform {
  required_version = "= 1.3.5"
  required_providers {
    aws = "= 4.44.0"
    random = {
      source  = "hashicorp/random"
      version = "~> 3.0"
    }
  }
}

Steps to Reproduce

• Run terraform.
• Uncomment "aws_route igw_route" in main.tf.
• Comment out "aws_route nat_route" in main.tf.
• Run terraform.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.