[FEATURE] Add networking support for AWS Outposts, Local Zones, and Wavelength Zones

[Brian969]

Describe the solution you'd like

• Add networking support for AWS Outposts, Local Zones, and Wavelength Zones
• Add ability to target customer created objects from VPC route tables

Describe alternatives you've considered

• Needs to be part of existing networking orchestration

Narrative

• Customer orders Outpost in their AWS account
• AWS installs Outpost into a customer's on-prem environment and associates it with a specific customer designated AWS account
  • providing an outpost-id, an outpost-arn, and an LZ (AZ in long format)
• AWS creates a Local Gateway (LGW) on the Outpost during Outpost installation
  • providing a LGW-id and LGW-rt-id
• Customer updates ASEA config file to create a new VPC defined at the OU level in the config file
  • Summary: Outpost will be deployed and owned in a new sub-account located in the Infrastructure OU (outpost-owner-account-key). VPC config will be defined in the ASEA in a new OU named Outpost_Dev (i.e. using the setting "deploy": "outpost-owner-account-key" and have "share-to-ou-accounts": true set on the subnets). Repeat for Test and Prod OU’s - if applicable
  • VPC will be created in the Outpost account, which can be thought of as a second shared-network account
  • VPC will contain existing regional objects like regional subnets, flow logs, route tables, and tgw-attachments, etc.
  • VPC will also contain new Outpost local subnets
  • VPC will reference a new LGW object (like it does for IGW, VGW, NATGW)
  • VPC will also contain route tables attached to Outpost based subnets with routes pointing to the lgw-id
• Customer will create a new account within the Outpost X OU (X=Dev/Test/Prod)
  • the VPC’s subnets will be shared to the new AWS account
  • (other share-to-options available like named accounts)
• Customer is expected to provide a valid outpost-arn, LZ (AZ), lgw-id, lgw-rt-id
  • Ideally fail the SM machine gracefully if these values are invalid, worst case proceed without indicating anything. Not desirable to leave stacks in rollback failed state
  • customer is expected to define the VPC to be created in a valid account (i.e. setting the “deploy” value within an ou VPC definition, or defining the VPC within a mandatory or workload account which owns the VPC).

Overall solution architecture

The below diagram (lower portion) documents the Outpost account, the Outpost, the Dev Outpost VPC, the recommended subnet layouts, etc.

Details

• VPC’s are regional (No impact)
  • VPC flowlogs are part of a VPC and also regional
• Route tables are regional (No impact)
  • associated with subnets and also have no zonal knowledge
• Subnets are zonal and can be local to an AZ, Local Zone, or Outpost (Impact)
  • Uses same subnet creation API for all three
  • Required API parameters supported in both AWS CLI and in AWS CDK today
  • For Outposts, requires supplying both the AZ-long-name and the Outpost ARN
  • need to add a new option config file value outpost-arn as part of the subnet object

    "subnets": [
    {
    "name": "Public",
    "definitions": [
    {
    "az": "a",
    "outpost-arn": "arn:aws:outposts:us-west-2:123456789012:outpost/op-0e12dade1280682b8",
    "route-table": "Public_Shared",
    "cidr": {
    "pool": "RFC6598b",
    "size": 26
    }
    },

• • In the cases of a Local Zone, the AZ-name can simply be replaced with with LZ-name
    • ASEA currently restricts az values to one of ‘a’ through ‘f’
    • ‘a’ would be transformed into ‘us-west-2a’ in us-west-2, ‘b’ to ‘ca-central-1b’ in ca-central-1
    • Do we a) relax the az definition field, or b) add a new “lz”: “us-west-2-lax-1a” field (i.e. “us-west-2-lax-1a” is the format expected by the API). If provided, we’d concat {region}{-}{lz}{az}. initial preference is b).
• Local Gateway is a new Outpost Construct (one per Outpost) (Impact)
  • created by the AWS team installing the Outpost and cannot be created by a customer
  • LGW has an associated LGW route table, also created by AWS and cannot be created or updated by a customer
  • the LGW needs to be ‘pointed to’ in routes in the route tables (which are regional constructs)
  • the LGW needs to be attached to the VPC
  • add a new parameter to a VPC “lgw”: “lgw-rt-id”, similar to the “igw”: true parameter
  • instead of creating and attaching the object, simply attach it
  • Required API parameters supported in both AWS CLI and in AWS CDK today
• Routes (Impact)
  • also regional, no zonal associations/knowledge
  • Routes need to point to a destination object type and are controlled by ASEA
  • Adding/updating routes uses same route creation API for all destination object types
  • targeting a LGW requires a slightly different API parameter
  • Required API parameters supported in both AWS CLI and in AWS CDK today
  • As currently the ASEA can only target ASEA created targets, we need a new config file variable to enable targeting customer provided/created object types
  • to be flexible, this should NOT be limited to LGW object types but any object types
  • ASEA currently has a route field named “target”
  • supported values of: IGW, firewall, pcx, TGW, s3, DynamoDB, NATGW_xxx (others may exist)
  • extend the target field to support the value “customer”
  • Option 1
    • add a new field called “type” which supports one of the following values: carrierGatewayId, egressOnlyInternetGatewayId, gatewayId, instanceId, localGatewayId, natGatewayId, networkInterfaceId, transitGatewayId, vpcEndpointId, vpcPeeringConnectionId (case insisitive). Example: “type”: “localGatewayId”,
    • add a new field called “target-id'” which would be expected to contain a value starting with lgw-, nat-, vpce-, igw-, pcx-, cagw-, eigw, i-, eni-, tgw-, vgw- and look something like: eigw-01eadbd45ecd7943f, tgw-0cc441aec1449c77d, vpce-0d9d14a6b496a618b, lgw-0cdc67d1ae6c75ff8. Example: “target-id”: “lgw-0cdc67d1ae6c75ff8”,
• Example:

  {
  "name": "RT_Name",
  "routes": [
  {
    "destination": "0.0.0.0/0",
    "target": "customer",
    "type": "localGatewayId",
    "target-id": "lgw-0cdc67d1ae6c75ff8",
  }
  ]
  },

• • Option 2
    • alternative: do not add a type or target-id field from option 1, and use the values for “type” field above as the config parameter name and the values for “target-id” field above as the config parameter value. Example: “ localGatewayId”: “lgw-0cdc67d1ae6c75ff8”,
• Example:

  {
  "name": "RT_Name",
  "routes": [
  {
    "destination": "0.0.0.0/0",
    "target": "customer",
    "localGatewayId": "lgw-0cdc67d1ae6c75ff8",
  }
  ]
  },

• RAM subnet/vpc sharing is regional (Not Impacted)
  • Create/associate resource share

Example CFN Stacks For Each of the above new activities:

Stack1 - Associate LGW to VPC (via the LGW-RT):

{
  "Resources": {
  "teststack1association": {
    "Type": "AWS::EC2::LocalGatewayRouteTableVPCAssociation",
    "Properties": {
      "LocalGatewayRouteTableId": "lgw-rtb-0aeee54ea969f0f11",
      "VpcId": "vpc-0963ea3ebdb560a2b",
      "Tags": [
        {
          "Key": "Accelerator",
          "Value": "ASEA"
        }
      ]
    }
  }
}
}

Stack2 - Add Route Table Entry Pointing to LGW:

{
    "Resources": {
"Teststack1": {
    "Type": "AWS::EC2::Route",
    "Properties": {
      "RouteTableId": "rtb-05da8adedf93db128",
      "DestinationCidrBlock": "192.168.1.0/24",
      "LocalGatewayId": "lgw-0cdc67d1ae2c86ff8"
    }
  }
}
}

Stack3 - Create Outpost Subnet:

{
  "Resources": {
    "TestSubnet": {
        "Type": "AWS::EC2::Subnet",
        "Properties": {
          "CidrBlock": "10.250.0.196/27",
          "VpcId": "vpc-0963ea3ebdb560a2b",
          "AvailabilityZone": "us-west-2a",
          "OutpostArn": "arn:aws:outposts:us-west-2:123456789012:outpost/op-0e56dade1930688c7",
          "Tags": [
            {
              "Key": "Accelerator",
              "Value": "ASEA"
            },
            {
              "Key": "Name",
              "Value": "My-Outpost-Subnet"
            }
          ]
        }
      }
  }
}

[rverma-dev]

Can we also include the capability of edge-association for the route tables?

[Brian969]

Hi, Can you elaborate on what you mean by edge-association? This PR (which is code complete) supports targeting any supported object type from a VPC route table, whether created by the ASEA or not. (i.e. 'egressOnlyInternetGatewayId', 'gatewayId', 'instanceId', 'localGatewayId', 'natGatewayId', 'networkInterfaceId', 'transitGatewayId', 'vpcEndpointId', 'vpcPeeringConnectionId',).