Closed phillipedwards closed 9 months ago
The diff related to gatewayId vs transitGatewayId appears to be a difference of opinion on types or polymorphism related. Eg- transitGateway is a type of gateway.
Something new about it? I get it also :|
Edit:
Change the from gateway_id
var to nat_gateway_id
and it will fix it.
Updating (repro-1574):
Type Name Status Info
+ pulumi:pulumi:Stack aws-ts-subnets-repro-1574 **creating failed (51s)** 1 error
+ ├─ aws:ec2:Vpc diff-vpc created (4s)
+ ├─ aws:ec2transitgateway:TransitGateway diff-example created (47s)
+ ├─ aws:ec2:Subnet diff-subnet created (1s)
+ ├─ aws:ec2transitgateway:VpcAttachment diff-=attach created (71s)
+ └─ aws:ec2:RouteTable diff-route-table **creating failed** 1 error
Diagnostics:
pulumi:pulumi:Stack (aws-ts-subnets-repro-1574):
error: update failed
aws:ec2:RouteTable (diff-route-table):
error: 1 error occurred:
* creating urn:pulumi:repro-1574::aws-ts-subnets::aws:ec2/routeTable:RouteTable::diff-route-table: 1 error occurred:
* error creating Route in Route Table (rtb-092ecf17a86f9364d) with destination (0.0.0.0/0): InvalidGatewayID.NotFound: The gateway ID 'tgw-0af86f9703986f3e2' does not exist
status code: 400, request id: dc83ecc2-6787-4522-8757-cfe14c171a31
Outputs:
gatewayId: "tgw-0af86f9703986f3e2"
Trying to reproduce this also noting that the first pulumi up
completely fails for this program.
Couple more notes. Reproduces on 6.12.3 for me. Indeed fixing the program like this can workaround the issue:
new aws.ec2.RouteTable("diff-route-table", {
vpcId: vpc.id,
routes: [{
cidrBlock: "0.0.0.0/0",
// gatewayId: gateway.id
transitGatewayId: gateway.id
}]
});
I've translated this program to TF.
resource "aws_vpc" "myvpc" {
cidr_block = "10.100.0.0/16"
}
resource "aws_ec2_transit_gateway" "tg" {
}
resource "aws_ec2_transit_gateway_vpc_attachment" "diffattach" {
subnet_ids = [aws_subnet.mysub.id]
transit_gateway_id = aws_ec2_transit_gateway.tg.id
vpc_id = aws_vpc.myvpc.id
}
resource "aws_subnet" "mysub" {
vpc_id = aws_vpc.myvpc.id
cidr_block = "10.100.1.0/24"
}
resource "aws_route_table" "example" {
vpc_id = aws_vpc.myvpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_ec2_transit_gateway.tg.id
}
}
It appears the same behavior is present in upstream 5.29.0 provider and TF CLI 1.6.3.
The first terraform apply fails with:
╷
│ Error: creating Route in Route Table (rtb-07e141eb97bd3b15c) with destination (0.0.0.0/0): InvalidGatewayID.NotFound: The gateway ID 'tgw-05f26a33b9e4b27bd' does not exist
│ status code: 400, request id: e9d50724-4877-45af-bafb-30920a42131a
│
│ with aws_route_table.example,
│ on infra.tf line 19, in resource "aws_route_table" "example":
│ 19: resource "aws_route_table" "example" {
│
╵
Subsequent Terraform apply gets stuck in a permanent diff with:
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
# aws_route_table.example will be updated in-place
~ resource "aws_route_table" "example" {
id = "rtb-0db9dc038995de7bb"
~ route = [
- {
- carrier_gateway_id = ""
- cidr_block = "0.0.0.0/0"
- core_network_arn = ""
- destination_prefix_list_id = ""
- egress_only_gateway_id = ""
- gateway_id = ""
- ipv6_cidr_block = ""
- local_gateway_id = ""
- nat_gateway_id = ""
- network_interface_id = ""
- transit_gateway_id = "tgw-05f26a33b9e4b27bd"
- vpc_endpoint_id = ""
- vpc_peering_connection_id = ""
},
+ {
+ carrier_gateway_id = ""
+ cidr_block = "0.0.0.0/0"
+ core_network_arn = ""
+ destination_prefix_list_id = ""
+ egress_only_gateway_id = ""
+ gateway_id = "tgw-05f26a33b9e4b27bd"
+ ipv6_cidr_block = ""
+ local_gateway_id = ""
+ nat_gateway_id = ""
+ network_interface_id = ""
+ transit_gateway_id = ""
+ vpc_endpoint_id = ""
+ vpc_peering_connection_id = ""
},
]
tags = {}
# (5 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
This appears somewhat documented https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table see note 2:
NOTE on gateway_id and nat_gateway_id:
The AWS API is very forgiving with these two attributes and the aws_route_table resource can be created with a NAT ID specified as a Gateway ID attribute. This will lead to a permanent diff between your configuration and statefile, as the API returns the correct parameters in the returned route table. If you're experiencing constant diffs in your aws_route_table resources, the first thing to check is whether or not you're specifying a NAT ID instead of a Gateway ID, or vice-versa.
It does not specifically mention transitGateway but I believe it behaves similarly here.
Looks like Pulumi documentation has also inherited this note: https://www.pulumi.com/registry/packages/aws/api-docs/ec2/routetable/
@iwahbe @phillipedwards based on the above investigation I suggest closing as "resolution/by-design". It will be relatively difficult for us to improve this behavior over what the upstream provider decided to be the by-design behavior. I'm open to other suggestions how to proceed here, perhaps we could enhance the Pulumi side documentation on the issue?
I agree with @t0yv0. The documentation correctly describes the observed behavior, and offers a path to resolve it. Upstream does the same thing.
@phillipedwards Does that work for you?
Closing. Please reopen if necessary.
As a developer, I want pulumi to correctly show whether or not my stack requires updates or not, so that I can correctly identify when I've made a change to my pulumi managed resources.
git clone https://gist.github.com/phillipedwards/f4604a2a5cbeb7761791f79ff2633859
pulumi up
pulumi up
Expected: No updates made to aws resources on second execution of
pulumi up
Actual:pulumi up
results in a diff for the route table's CIDR block and gateway ID