aws-ia / terraform-aws-vpc

AWS VPC Module
https://registry.terraform.io/modules/aws-ia/vpc/aws/latest
Apache License 2.0
82 stars 89 forks source link

Trying to get IPAM to Allocate a Secondary CIDR Fails; CIDR Already Allocated to the Primary VPC is Used #146

Open imaginaryCorn opened 4 months ago

imaginaryCorn commented 4 months ago

Hello! I am trying to add another CIDR and subnets to a VPC. Both the VPC and the secondary are made using this module, and both use IPAM to get their IPv4 CIDRs.

I've discovered that the module is retrieving the CIDR block of the existing VPC when creating the aws_vpc_ipv4_cidr_block_association.secondary[0], and not a new one from IPAM:

Screenshot 2024-02-29 at 10 09 53 AM

The config seems to conditionally use the CIDR of the primary VPC if the var.cidr_block is not set and a VPC is not being created, but it does not ignore this if IPAM settings are set: https://github.com/aws-ia/terraform-aws-vpc/blob/main/data.tf#L94C3-L95C78

Therefore, aws_vpc_ipv4_cidr_block_association.secondary[0] is given both a CIDR that has already been allocated and the IPAM pool ID, the latter then gets ignored by Terraform or AWS and the apply fails because of it trying to allocate the already allocated CIDR:

Screenshot 2024-02-29 at 10 18 27 AM

Is this intended behaviour? Is IPAM not meant to be used with this module for secondary CIDR's?

My code with omissions:

data "aws_vpc_ipam_pool" "engineering" {
  filter {
    name   = "description"
    values = ["us-west-2-prod"]
  }

  filter {
    name   = "address-family"
    values = ["ipv4"]
  }
}

module "vpc" {
  source  = "aws-ia/vpc/aws"
  version = "4.4.2"

  name                             = "***"
  vpc_ipv4_ipam_pool_id            = data.aws_vpc_ipam_pool.engineering.id
  vpc_ipv4_netmask_length          = 22
  vpc_egress_only_internet_gateway = false
  az_count                         = 3

  subnets = {
    public = {
      name_prefix               = "***"
      netmask                   = 28
      nat_gateway_configuration = "all_azs" # options: "single_az", "none"
    }

    private = {
      name_prefix             = "***"
      netmask                 = 24
      connect_to_public_natgw = true
    }

    # Manually create az_count x additional subnets without recalculating the existing subnets
    # (https://github.com/aws-ia/terraform-aws-vpc#updating-a-vpc-with-new-or-removed-subnets):
    private_additional = {
      name_prefix             = "***"
      cidrs                   = ["10.160.3.64/26", "10.160.3.128/26", "10.160.3.192/26"]
      connect_to_public_natgw = true
    }
  }

  vpc_flow_logs = {
    log_destination_type = "s3"
    log_destination      = "***"
    destination_options = {
      file_format                = "plain-text"
      hive_compatible_partitions = true
      per_hour_partition         = true
    }
  }
}

module "secondary" {
  source  = "aws-ia/vpc/aws"
  version = "4.4.2"

  name                    = "***"
  az_count                = 3
  vpc_ipv4_ipam_pool_id   = data.aws_vpc_ipam_pool.engineering.id
  vpc_ipv4_netmask_length = 22

  vpc_secondary_cidr       = true
  vpc_id                   = module.vpc.vpc_attributes.id
  vpc_secondary_cidr_natgw = module.vpc.natgw_id_per_az

  subnets = {
    private = {
      name_prefix             = "***"
      netmask                 = 24
      connect_to_public_natgw = true
    }

    private_additional = {
      name_prefix             = "***"
      netmask                 = 26
      connect_to_public_natgw = true
    }
  }
}