Be able to change AZ count on already deployed VPC (constructs need to be able to keep state)

[ranrotx]

When (in my case) increasing the maxAzs specified using the VPC construct from 3 to 4 and running cdk deploy, the CloudFormation stack update fails with an error code of InvalidSubnet.Conflict;

Reproduction Steps

Changing

const vpc = new ec2.Vpc(this, 'VPC', {
      cidr: "172.21.0.0/16",
      maxAzs: 3,
      natGateways: 1,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'public',
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 24,
          name: 'private',
          subnetType: ec2.SubnetType.PRIVATE,
        }
      ]
    });

to

const vpc = new ec2.Vpc(this, 'VPC', {
      cidr: "172.21.0.0/16",
      maxAzs: 4,
      natGateways: 1,
      subnetConfiguration: [
        {
          cidrMask: 24,
          name: 'public',
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 24,
          name: 'private',
          subnetType: ec2.SubnetType.PRIVATE,
        }
      ]
    });

produces the error below.

Error Log

1/28 | 5:55:19 PM | CREATE_FAILED        | AWS::EC2::Subnet                      | VPC/publicSubnet4/Subnet (VPCpublicSubnet4Subnet46529D45) The CIDR '172.21.3.0/24' conflicts with another subnet (Service: AmazonEC2; Status Code: 400; Error Code: InvalidSubnet.Conflict; Request ID: 41f1d4ed-b249-4cfa-bb1e-c0148a4ddc30)
        new Subnet (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1373:20)
        \_ new PublicSubnet (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1588:5)
        \_ /Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1239:32
        \_ Array.forEach (<anonymous>)
        \_ Vpc.createSubnetResources (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1221:28)
        \_ Vpc.createSubnets (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1210:12)
        \_ new Vpc (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/@aws-cdk/aws-ec2/lib/vpc.ts:1076:10)
        \_ new CdkVpcTransitgatewayStack (/Users/username/workspace/cdk-vpc-transitgateway/lib/cdk-vpc-transitgateway-stack.ts:8:17)
        \_ Object.<anonymous> (/Users/username/workspace/cdk-vpc-transitgateway/bin/cdk-vpc-transitgateway.ts:10:1)
        \_ Module._compile (internal/modules/cjs/loader.js:1147:30)
        \_ Module.m._compile (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/ts-node/src/index.ts:814:23)
        \_ Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
        \_ Object.require.extensions.<computed> [as .ts] (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/ts-node/src/index.ts:817:12)
        \_ Module.load (internal/modules/cjs/loader.js:996:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:896:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
        \_ main (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/ts-node/src/bin.ts:226:14)
        \_ Object.<anonymous> (/Users/username/workspace/cdk-vpc-transitgateway/node_modules/ts-node/src/bin.ts:485:3)
        \_ Module._compile (internal/modules/cjs/loader.js:1147:30)
        \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
        \_ Module.load (internal/modules/cjs/loader.js:996:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:896:14)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
        \_ /usr/local/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

Environment

• CLI Version : 1.27.0 (build a98c0b3)
• Framework Version: node --version v13.10.1
• OS : macOS 10.14.6
• Language : us-english

Other

At first glance, it appears that on subsequent deployments the VPC construct is not aware of what CIDR ranges may already be in-use.

---

This is :bug: Bug Report

[rix0rrr]

This is true, and solving this will require CDK applications to be able to keep state between runs.

We can use this use case as a motivating example for that feature.

[joeyaurel]

Got the same issue today, while setting my maxAzs from 1 to 2 for the CIDR 10.1.0.0/16 using the CLI in version 1.39.0.

Any updates on this? @rix0rrr

[klima-markus]

It looks like that the same happens if you increase the number of subnets in the following way

const vpc = new Vpc(this, `base-vpc`, {
    cidr: "11.0.0.0/16",
    subnetConfiguration: [
        {
            name: "public",
            subnetType: SubnetType.PUBLIC,
        },
        {
            name: "myPrivateSubnet",
            subnetType: SubnetType.PRIVATE,
        },
    ],
    maxAzs,
});

if you add another subnet after a deployment like

const vpc = new Vpc(this, `base-vpc`, {
    cidr: "11.0.0.0/16",
    subnetConfiguration: [
        {
            name: "public",
            subnetType: SubnetType.PUBLIC,
        },
        {
            name: "myPrivateSubnet",
            subnetType: SubnetType.PRIVATE,
        },
        {
            name: "myOtherPrivateSubnet",
            subnetType: SubnetType.PRIVATE,
        },
    ],
    maxAzs,
});

=> conflicts with another subnet error

[rix0rrr]

Related to https://github.com/aws/aws-cdk/issues/5927 (and other issues in that area)

[carlomorelli]

this happened to me today, when reducing maxazs from 2 to 1. Is there a workaround to do this ?

[sunshineo]

Happened to me when set maxAzs to 1. I had to destroy the stack and create again. Good thing we have nothing else in the VPC. I guess in production, one have to create a new VPC and move everything

[chadnash]

This is true, and solving this will require CDK applications to be able to keep state between runs.

We can use this use case as a motivating example for that feature.

that state is held in AWS and it should be cloudformations to job to cope

[patones]

suffering from the same issue. increase or decrease :(

[Jordhan-Carvalho]

Having the same problem adding another subnet

[tobias-nawa]

I'm trying to remove my Isolated subnet and am running into the same issue. CDK deployment wants to create all subnets from scratch, rather than just removing the one I want to get rid of. This causes the issue with the CIDRs already in use.

This is the code where I just removed the first subnet:

            subnet_configuration=[
                ec2.SubnetConfiguration(
                    name="Isolated",
                    cidr_mask=24,
                    reserved=False,
                    subnet_type=ec2.SubnetType.ISOLATED,
                ),
                ec2.SubnetConfiguration(
                    name="Private",
                    cidr_mask=22,
                    reserved=False,
                    subnet_type=ec2.SubnetType.PRIVATE,
                ),
                ec2.SubnetConfiguration(
                    name="Public",
                    cidr_mask=23,
                    reserved=False,
                    subnet_type=ec2.SubnetType.PUBLIC,
                ),
            ],

[SoccerBoyMalloy]

Different but related, I received the same error when trying to change NatGateways from 1 to 0.

[carlomorelli]

It's unfortunate that devs don't give priority to fix this Vpc construct; without these problems fixed, relying on it is very dangerous.

[benjaminwhire]

Is there any workaround? I encountered the same issue without tweaking the AZ.

[zachgoll]

I've encountered this issue several times. It doesn't really matter what you're changing--if you attempt to change the subnet structure of an already-created VPC with CDK, you'll get this error.

Has anyone come up with a workaround, or even an order of operations for making changes to a VPC? My thought is that the previous VPC subnets would have to be completely destroyed prior to an update (since the CIDR reservations are not editable, even in the console)

[damshenas]

+1

[comcalvi]

Related to the tracking state between deployments issue: #13676.

[zippocage]

Would also like to be able to change AZ count.

[joeyvmason]

Also experiencing this issue. Not uncommon to need to tweak VPC and if that means having to tear down your entire stack that is a huge dealbreaker for using this in production

[mikelane]

I just ran into this issue today too. Would be nice to not have to destroy the stack in order to update this.

[bataras]

same issue here. has aws ever phsically added an AZ to a Region? I would think if they did that, lots of CDK users would not be happy