Wrong IP firewall port rules for aws.lightsail.InstancePublicPorts after first change

[ceefour]

Wrong IP firewall port rules for aws.lightsail.InstancePublicPorts after first change

Expected behavior

IP Firewall is modified exactly according to portInfos

Current behavior

During creation, portInfos work exactly. However, subsequent modifications have erratic behavior: one or more missing rules.

Steps to reproduce

0. Create any Lightsail instance, e.g.

const mysqlFusionAuth = new aws.lightsail.Instance("mysql-sg", {
  name: "mysql-sg",
  availabilityZone: "ap-southeast-1a",
  keyPairName: "default",
  bundleId: "nano_2_0",
  blueprintId: "ubuntu_20_04",
});

1. Create an aws.lightsail.InstancePublicPorts, then up this one:

  portInfos: [
    {
      protocol: "tcp",
      fromPort: 22,
      toPort: 22,
      cidrs: ["0.0.0.0/0"],
    },
    {
      protocol: "tcp",
      fromPort: 3306,
      toPort: 3306,
      cidrs: ["172.31.0.0/16"],
    },
  ],

2. Now change it to:

  portInfos: [
    {
      protocol: "tcp",
      fromPort: 22,
      toPort: 22,
      cidrs: ["172.31.0.0/16"],
    },
    {
      protocol: "tcp",
      fromPort: 3306,
      toPort: 3306,
      cidrs: ["172.31.0.0/16"],
    },
  ],

3. The above will cause all rules to be deleted, leaving the Lightsail instance not connectable in any way.
4. Change it again to:

  portInfos: [
    {
      protocol: "tcp",
      fromPort: 22,
      toPort: 22,
      cidrs: ["0.0.0.0/0"],
    },
    {
      protocol: "tcp",
      fromPort: 3306,
      toPort: 3306,
      cidrs: ["172.31.0.0/16"],
    },
  ],

5. The above will correctly open port 22 from anywhere, however deleting the port rule for 3306.

Context (Environment)

Region: ap-southeast-1

Affected feature

• aws.lightsail
• area/providers

[mskutin]

Got into this too. Initial creation of the instance is fine, but every subsequent pulumi up creates a new diff and completely removes all IPV4 rules, although portInfos array is untouched.

const publicPorts = new aws.lightsail.InstancePublicPorts(`${vpnPrefix}-public-ports`, {
  instanceName: vpnInstance.name,
  portInfos: [
    {
      protocol: 'tcp',
      fromPort: 80,
      toPort: 80,
    },
    {
      protocol: 'tcp',
      fromPort: 22,
      toPort: 22,
    },
    // Outline Management port
    {
      protocol: 'tcp',
      fromPort: 42207,
      toPort: 42207,
    },
    // Outline Access Keys
    {
      protocol: 'tcp',
      fromPort: 32321,
      toPort: 32321,
    },
    // Outline Access Keys
    {
      protocol: 'udp',
      fromPort: 32321,
      toPort: 32321,
    },
  ],
});

~ portInfos: [
          ~ [0]: {
                  ~ fromPort: 80 => 80
                  ~ protocol: "tcp" => "tcp"
                  ~ toPort  : 80 => 80
                }
          ~ [1]: {
                  ~ fromPort: 22 => 22
                  ~ protocol: "tcp" => "tcp"
                  ~ toPort  : 22 => 22
                }
          ~ [2]: {
                  ~ fromPort: 42207 => 42207
                  ~ protocol: "tcp" => "tcp"
                  ~ toPort  : 42207 => 42207
                }
          ~ [3]: {
                  ~ fromPort: 32321 => 32321
                  ~ protocol: "udp" => "tcp"
                  ~ toPort  : 32321 => 32321
                }
          ~ [4]: {
                  ~ fromPort: 32321 => 32321
                  ~ protocol: "tcp" => "udp"
                  ~ toPort  : 32321 => 32321
                }
        ]

[lukehoban]

The issue appears to be that this resource in the upstream provider does not handle create-before-replace correctly. When a replacement happens, it will create the new rules, then delete the "old" ones, which will inadvertently delete the new ones as well due to incorrect behaviour of the provider.

In Pulumi, you can workaround this by applying deleteBeforeReplace to the resource. That will ensure that the delete happens first, so that it doesn't undo the create. I verified that this works for the original repro case above:

const mysqlFusionAuth = new aws.lightsail.Instance("mysql-sg", {
    name: "mysql-sg",
    availabilityZone: "us-west-2a",
    keyPairName: "foobar",
    bundleId: "nano_2_0",
    blueprintId: "ubuntu_20_04",
});

const ports = new aws.lightsail.InstancePublicPorts("ports", {
    instanceName: mysqlFusionAuth.name,
    portInfos: [
        {
            protocol: "tcp",
            fromPort: 22,
            toPort: 22,
            cidrs: ["0.0.0.0/0"],
        },
        {
            protocol: "tcp",
            fromPort: 3306,
            toPort: 3306,
            cidrs: ["172.31.0.0/16"],
        },
    ],
}, { deleteBeforeReplace: true})  // <----- Use delete before replace mode

This is slightly unfortunate though, as it means it is impossible to change the port configuration without some downtime (between the delete and the create). The right thing would be for the upstream provider to support create before delete mode as well (which almost all resources do).

We could in principle force this resource to always use deleteBeforeReplace mode. I'm not certain that is the right choice, but it might help avoid potential issues here.