Provider produced inconsistent final plan When expanding the plan produced an invalid new value for .route: planned set element

acvir commented 2 years ago

Error Output

Provider produced inconsistent final plan , When expanding the plan for module.subnets[0].aws_route_table.route_tables[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .route: planned set element

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.7 Provider hashicorp/aws v4.6.0

Affected Resource(s)

aws_vpc
aws_vpc_subnets
aws_vpc_routetables

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

subnet configration
-------------------
locals {
  primary_vpc_db_sub_cidrs   = split(",", data.aws_ssm_parameter.primary_vpc_db_subnet.value)
  secondary_vpc_db_sub_cidrs = split(",", data.aws_ssm_parameter.secondary_vpc_db_subnet.value)
}

#--------------------------------------------------------------
# Creates subnets within the provided vpc in the primary region
#--------------------------------------------------------------
resource "aws_subnet" "primary_subnet_db" {
  provider = aws.primary_region
  depends_on = [
    data.aws_ssm_parameter.primary_vpc_db_subnet
  ]
  count = var.primary_db_azs_count

  vpc_id            = var.vpc_ids[0]
  cidr_block        = local.primary_vpc_db_sub_cidrs[count.index]
  availability_zone = var.primary_db_azs_list[count.index]

  map_public_ip_on_launch = false

  tags = merge(
    {
      Name             = "DB Subnet - ${var.primary_db_azs_list[count.index]}"
      Tier             = "Database"
      AvailabilityZone = var.primary_db_azs_list[count.index]
    },
    var.tags
  )
}

#----------------------------------------------------------------
# Creates subnets within the provided vpc in the secondary region
#----------------------------------------------------------------
resource "aws_subnet" "secondary_subnet_db" {
  provider = aws.secondary_region
  depends_on = [
    data.aws_ssm_parameter.secondary_vpc_db_subnet
  ]
  count = var.secondary_db_azs_count

  vpc_id            = var.vpc_ids[1]
  cidr_block        = local.secondary_vpc_db_sub_cidrs[count.index]
  availability_zone = var.secondary_db_azs_list[count.index]

  map_public_ip_on_launch = false

  tags = merge(
    {
      Name             = "DB Subnet - ${var.secondary_db_azs_list[count.index]}"
      Tier             = "Database"
      AvailabilityZone = var.secondary_db_azs_list[count.index]
    },
    var.tags
  )
}

-------

route table 
------------
# Creating route tables for the db subnets in the primary region
resource "aws_route_table" "primary_db_route_tables" {
  provider   = aws.primary_region
  depends_on = [aws_subnet.primary_subnet_db]
  count      = var.primary_db_azs_count

  vpc_id = var.vpc_ids[0]
  route {
    destination_prefix_list_id = var.primary_prefix_id
    transit_gateway_id         = data.aws_ec2_transit_gateway.tgw_primary.id
  }

  tags = merge(
    {
      Name = "DB-RT-${var.primary_db_azs_list[count.index]}"
    },
    var.tags
  )
}
# Associating the route table with the subnet ids in the primary region
resource "aws_route_table_association" "primary_db_rt_subnet_links" {
  provider = aws.primary_region
  depends_on = [
    aws_subnet.primary_subnet_db,
    aws_route_table.primary_db_route_tables
  ]
  count          = var.primary_db_azs_count
  subnet_id      = aws_subnet.primary_subnet_db[count.index].id
  route_table_id = aws_route_table.primary_db_route_tables[count.index].id
}
#=====================================================================
# Creating route tables for the db subnets in the secondary region
resource "aws_route_table" "secondary_db_route_tables" {
  provider   = aws.secondary_region
  depends_on = [aws_subnet.secondary_subnet_db]
  vpc_id     = var.vpc_ids[1]
  route {
    destination_prefix_list_id = var.secondary_prefix_id
    transit_gateway_id         = data.aws_ec2_transit_gateway.tgw_secondary.id
  }
  count = var.secondary_db_azs_count

  tags = merge(
    {
      Name = "DB-RT-${var.secondary_db_azs_list[count.index]}"
    },
    var.tags
  )
}
# Associating the route table with the subnet ids in the secondary region
resource "aws_route_table_association" "secondary_db_rt_subnet_links" {
  provider = aws.secondary_region
  depends_on = [
    aws_subnet.secondary_subnet_db,
    aws_route_table.secondary_db_route_tables
  ]
  count          = var.secondary_db_azs_count
  subnet_id      = aws_subnet.secondary_subnet_db[count.index].id
  route_table_id = aws_route_table.secondary_db_route_tables[count.index].id
}
#====================================================
# Adding routes to the NAT Gateway in DB Route Tables
#====================================================
# Adding Route to NAT Gatway in the Priary Region's DB Route Tables
resource "aws_route" "primary_nat_db_routes" {
  provider = aws.primary_region
  depends_on = [
    aws_route_table.primary_db_route_tables
  ]
  count                  = var.primary_db_azs_count
  route_table_id         = aws_route_table.primary_db_route_tables[count.index].id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = var.primary_nat_ids[count.index]
}

# Adding Route to NAT Gatway in the Secondary Region's DB Route Tables
resource "aws_route" "secondary_nat_db_routes" {
  provider = aws.secondary_region
  depends_on = [
    aws_route_table.secondary_db_route_tables
  ]
  count                  = var.secondary_db_azs_count
  route_table_id         = aws_route_table.secondary_db_route_tables[count.index].id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = var.secondary_nat_ids[count.index]
}

Debug Output

Panic Output

Error: Provider produced inconsistent final plan │ │ When expanding the plan for │ module.db-subnets[0].aws_route_table.secondary_route_tables[0] to │ include new values learned so far during apply, provider │ "registry.terraform.io/hashicorp/aws" produced an invalid new value for │ .route: planned set element │ cty.ObjectVal(map[string]cty.Value{"carrier_gateway_id":cty.StringVal(""), │ "cidr_block":cty.StringVal(""), "core_network_arn":cty.StringVal(""), │ "destination_prefix_list_id":cty.StringVal("pl-079eaab23a45a8974"), │ "egress_only_gateway_id":cty.StringVal(""), "gateway_id":cty.StringVal(""), │ "instance_id":cty.StringVal(""), "ipv6_cidr_block":cty.StringVal(""), │ "local_gateway_id":cty.StringVal(""), "nat_gateway_id":cty.StringVal(""), │ "network_interface_id":cty.StringVal(""), │ "transit_gateway_id":cty.UnknownVal(cty.String), │ "vpc_endpoint_id":cty.StringVal(""), │ "vpc_peering_connection_id":cty.StringVal("")}) does not correlate with any │ element in actual. │ │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker.

Expected Behavior

Update the vpc subnets and route tables , when updating via terrform using AWS customizations-pipeline CodePipeline

Actual Behavior

Steps to Reproduce

provision the vpc with subnets and route tables
update the subnets with new values
perform a release change from AWS customizations-pipeline CodePipeline

terraform apply

Important Factoids

References

0000

justinretzolk commented 2 years ago

Hey @acvir 👋 Thank you for taking the time to raise this! So that we have all of the necessary information in order to look into this, can you supply the Terraform configuration as well as the debug logs (if possible, and redacted as needed)?

acvir commented 2 years ago

Hey @justinretzolk Updated the terrraform config files section above.

amrinder145 commented 2 months ago

Hi @justinretzolk Do we have any updates on this?

amrinder145 commented 2 months ago

I am also getting similar error:

Error: Provider produced inconsistent final plan │ │ When expanding the plan for │ module.test_vpc.aws_route_table.public["10.20.11.0/28"] to include new │ values learned so far during apply, provider │ "registry.terraform.io/hashicorp/aws" produced an invalid new value for │ .route: actual set element │ cty.ObjectVal(map[string]cty.Value{"carrier_gateway_id":cty.NullVal(cty.String), │ "cidr_block":cty.NullVal(cty.String), │ "core_network_arn":cty.NullVal(cty.String), │ "destination_prefix_list_id":cty.NullVal(cty.String), │ "egress_only_gateway_id":cty.NullVal(cty.String), │ "gateway_id":cty.StringVal("local"), │ "ipv6_cidr_block":cty.StringVal("2600:1f11:db7:400::/56"), │ "local_gateway_id":cty.NullVal(cty.String), │ "nat_gateway_id":cty.NullVal(cty.String), │ "network_interface_id":cty.NullVal(cty.String), │ "transit_gateway_id":cty.NullVal(cty.String), │ "vpc_endpoint_id":cty.NullVal(cty.String), │ "vpc_peering_connection_id":cty.NullVal(cty.String)}) does not correlate │ with any element in plan. │ │ This is a bug in the provider, which should be reported in the provider's │ own issue tracker.

hashicorp / terraform-provider-aws