[Bug]: expanding allowed prefixes aws_dx_gateway_association causes downtime

[marcincuber]

Terraform Core Version

1.4.1

AWS Provider Version

4.58

Affected Resource(s)

• aws_dx_gateway_association
• aws_ec2_transit_gateway_route_table_association

Expected Behavior

When updating aws_dx_gateway_association with an additional allowed_prefix. Gateway association should be updated in-place with zero impact on other prefixes.

Actual Behavior

Update to aws_dx_gateway_association resource causes aws_ec2_transit_gateway_route_table_association replacement which is disabling all the traffic through the dx gateway.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_dx_gateway" "lnd_eng_1" {
  name            = "LDN_IXN_ENG_1"
  amazon_side_asn = "6000"
}

resource "aws_dx_gateway_association" "lnd_eng_1" {
  dx_gateway_id         = aws_dx_gateway.lnd_eng_1.id
  associated_gateway_id = aws_ec2_transit_gateway.main_router.id

  allowed_prefixes = local.dx_gateway_allowed_prefixes
}

resource "aws_ec2_transit_gateway_route_table_association" "dx_gateway_lnd_eng_1" {
  transit_gateway_attachment_id  = data.aws_ec2_transit_gateway_attachment.lnd_eng_1.transit_gateway_attachment_id
  transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.main_router.id
}

Plan Output

  ~ resource "aws_dx_gateway_association" "lnd_eng_1" {
      ~ allowed_prefixes                    = [
          + "10.10.10.0/23",
            # (4 unchanged elements hidden)
        ]
        id                                  = "ga-46208b72-12345"
        # (6 unchanged attributes hidden)
    }

    # aws_ec2_transit_gateway_route_table_association.dx_gateway_lnd_eng_1 must be replaced
-/+ resource "aws_ec2_transit_gateway_route_table_association" "dx_gateway_lnd_eng_1" {
      ~ id                             = "tgw-rtb-3445_tgw-attach-1234" -> (known after apply)
      ~ resource_id                    = "46208b72-f184-12345" -> (known after apply)
      ~ resource_type                  = "direct-connect-gateway" -> (known after apply)
      ~ transit_gateway_attachment_id  = "tgw-attach-123434" # forces replacement -> (known after apply)
        # (1 unchanged attribute hidden)
    }

Steps to Reproduce

Implement aws_dx_gateway_association which is associated with transit gateway aws_ec2_transit_gateway_route_table_association.dx_gateway

Important Factoids

aws_ec2_transit_gateway_route_table_association.dx_gateway_lnd_eng_1 must be replaced action shouldn't be happening as allowed prefixes can be updated in place.

References

Related docs -> https://docs.aws.amazon.com/directconnect/latest/UserGuide/allowed-to-prefixes.html

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @marcincuber 👋 Thank you for taking the time to raise this! So that we have all of the necessary information in order to look into this, can you supply a sample Terraform configuration that exhibits this behavior?

[marcincuber]

@justinretzolk I updated the issue with sample configuration.

[per-lind]

Any news on this issue? I have the same problem and it makes it really hard to work with the transitgateway together with directconnect.

[sathishsrinu]

@marcincuber - did you find any solution on this?

[stanleycyk]

this is a critical feature and the downtime created by Terraform is unacceptable...

[stanleycyk]

@justinretzolk May I know if there is any update to this issue? This creates service interruption to all DX associations when we add new prefixes...