search

pulumi / pulumi-awsx

AWS infrastructure best practices in component form!
https://www.pulumi.com/docs/guides/crosswalk/aws/
Apache License 2.0
219 stars 104 forks source link

After version update VPC subnets creating differently and causing changes #1152

Closed benswinburne closed 10 months ago

benswinburne commented 10 months ago

What happened?

I have been in the unfortunate position where I'm having to upgrade a very old Pulumi stack.

"@pulumi/aws": "^3.19.2" -> "^6.8.0"
"@pulumi/awsx": "^0.22.0" -> "^2.1.1"
"@pulumi/pulumi": "^2.15.4" -> "^3.93.0"

Having updated the codebase to run, everything is fine except for the subnet definitions in the VPC. Despite no changes to my code, it now breaks and I can't preview/deploy my IaC.

Error: The following subnets overlap with at least one other subnet. Make the CIDR for the VPC larger, reduce the size of the subnets per AZ, or use less Availability Zones:

  1. vpc-db-1: 10.0.64.0/19
  2. vpc-private-2: 10.0.64.0/19
  3. vpc-db-2: 10.0.128.0/19
  4. vpc-private-3: 10.0.128.0/19

Given that there are no changes to the subnet definitions, presumably there should be no changes detected by Pulumi?

 export const vpc = new awsx.ec2.Vpc('vpc', {
   numberOfAvailabilityZones: 3,
-  numberOfNatGateways: 0,
+  natGateways: {
+    strategy: awsx.ec2.NatGatewayStrategy.None,
+  },
   cidrBlock: '10.0.0.0/16',
-  subnets: [
-    { type: 'private', cidrMask: 19 },
-    { type: 'isolated', cidrMask: 19, name: 'db' },
-    { type: 'public', cidrMask: 20 },
+  subnetSpecs: [
+    { type: awsx.ec2.SubnetType.Private, cidrMask: 19 },
+    { type: awsx.ec2.SubnetType.Isolated, cidrMask: 19, name: 'db' },
+    { type: awsx.ec2.SubnetType.Public, cidrMask: 20 },
   ],
 });

Are there any workarounds so I can just hardcode the original subnets in so they don't change.

Example

Install the following dependencies

"@pulumi/aws": "^3.19.2" "@pulumi/awsx": "^0.22.0" "@pulumi/pulumi": "^2.15.4"

import * as aws from '@pulumi/aws';
import * as awsx from '@pulumi/awsx';
import * as pulumi from '@pulumi/pulumi';

 export const vpc = new awsx.ec2.Vpc('vpc', {
   numberOfAvailabilityZones: 3,
   numberOfNatGateways: 0,
   cidrBlock: '10.0.0.0/16',
   subnets: [
     { type: 'private', cidrMask: 19 },
     { type: 'isolated', cidrMask: 19, name: 'db' },
     { type: 'public', cidrMask: 20 },
   ],
 });
pulumi up -y

Update dependencies to

"@pulumi/aws": "^6.8.0"
"@pulumi/awsx": "^2.1.1"
"@pulumi/pulumi": "^3.93.0"

Change VPC definition to 

import * as aws from '@pulumi/aws';
import * as awsx from '@pulumi/awsx';
import * as pulumi from '@pulumi/pulumi';

 export const vpc = new awsx.ec2.Vpc('vpc', {
   numberOfAvailabilityZones: 3,
   natGateways: {
     strategy: awsx.ec2.NatGatewayStrategy.None,
   },
   cidrBlock: '10.0.0.0/16',
   subnetSpecs: [
     { type: awsx.ec2.SubnetType.Private, cidrMask: 19 },
     { type: awsx.ec2.SubnetType.Isolated, cidrMask: 19, name: 'db' },
     { type: awsx.ec2.SubnetType.Public, cidrMask: 20 },
   ],
 });
pulumi up

Observe problem.

Output of pulumi about

CLI
Version      3.93.0
Go Version   go1.21.3
Go Compiler  gc

Plugins
NAME    VERSION
aws     6.8.0
awsx    2.1.1
docker  4.4.5
docker  3.6.1
nodejs  unknown
random  4.14.0

Host
OS       darwin
Version  13.4.1
Arch     arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v20.5.1'

Backend
Name           macbook.local
URL            s3://removed-for-obvious-reasons
User           myuser
Organizations
Token type     personal

Dependencies:
NAME                              VERSION
@aws-sdk/client-athena            3.450.0
@aws-sdk/client-glue              3.450.0
@aws-sdk/client-s3                3.450.0
@pulumi/aws                       6.8.0
@pulumi/awsx                      2.1.1
@pulumi/pulumi                    3.93.0
@pulumi/random                    4.14.0
@sentry/node                      5.15.5
aws-sdk                           2.1495.0
knex                              0.20.13
line-reader                       0.4.0
pg                                8.0.0
pipeline-pipe                     0.3.0
psl                               1.8.0
readline-transform                1.0.0
uuid                              7.0.3
@types/aws-lambda                 8.10.126
@types/aws-sdk                    2.7.0
@types/node                       20.9.0
@types/psl                        1.1.0
@types/readline-transform         1.0.0
@types/uuid                       7.0.2
@typescript-eslint/eslint-plugin  6.11.0
@typescript-eslint/parser         6.11.0
eslint                            8.53.0
eslint-config-prettier            9.0.0
eslint-plugin-import              2.29.0
eslint-plugin-prettier            5.0.1
nodemon                           2.0.2
prettier                          3.1.0
typescript                        5.2.2

Additional context

Obviously the VPC is attached to many resources including databases which are a huge rigmarole to replace, and the the VPC ID is referred to by other stacks etc so replacing the VPC would be far from ideal

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

danielrbradley commented 10 months ago

Hi @benswinburne sorry you're hitting this issue while upgrading. There was indeed a breaking change in the original 1.0 where there was a change to the layout of the subnets. You can still use the old behaviour for now by changing your import to:

import * as awsx from '@pulumi/awsx/classic';

This should be resolved once the following enhancements have been completed to allow more control over the layout:

Out of interest, what is the current layout of your deployed subnets?

Here's what I think the 1.x code is generating for you:

      [
        {
          azName: 'us-east-1a',
          cidrBlock: '10.0.0.0/19',
          type: 'Private',
          subnetName: 'vpcName-private-1',
        }, {
          azName: 'us-east-1a',
          cidrBlock: '10.0.32.0/20',
          type: 'Public',
          subnetName: 'vpcName-public-1',
        }, {
          azName: 'us-east-1a',
          cidrBlock: '10.0.64.0/19',
          type: 'Isolated',
          subnetName: 'vpcName-db-1',
        }, {
          azName: 'us-east-2b',
          cidrBlock: '10.0.64.0/19',
          type: 'Private',
          subnetName: 'vpcName-private-2',
        }, {
          azName: 'us-east-2b',
          cidrBlock: '10.0.96.0/20',
          type: 'Public',
          subnetName: 'vpcName-public-2',
        }, {
          azName: 'us-east-2b',
          cidrBlock: '10.0.128.0/19',
          type: 'Isolated',
          subnetName: 'vpcName-db-2',
        }, {
          azName: 'us-east-2c',
          cidrBlock: '10.0.128.0/19',
          type: 'Private',
          subnetName: 'vpcName-private-3',
        }, {
          azName: 'us-east-2c',
          cidrBlock: '10.0.160.0/20',
          type: 'Public',
          subnetName: 'vpcName-public-3',
        }, {
          azName: 'us-east-2c',
          cidrBlock: '10.0.192.0/19',
          type: 'Isolated',
          subnetName: 'vpcName-db-3',
        }
      ]
benswinburne commented 10 months ago

Hi @danielrbradley

Thanks for the super quick response.

My subnet layout is not quite as you described, but similar.

image

That said, changing to use awsx/classic has resolved the problem for now. That has allowed me to get on my way and I'll be able to shift this infrastructure around now.

Thanks very much for your help. I'll keep an eye on #1135, too.

danielrbradley commented 10 months ago

This is now fixed as part of #1135

There is a new subnetStrategy field on the VPC which can be set to "Auto" rather than the default "Legacy" mode. This will create the subnets in the same order as they were defined and also allows the adding of custom reserved blocks by adding a subnet spec with type "Unused".