Enable user-supplied Viewer Nodes CIDR

[chelma]

Description

As part of implementing (https://github.com/arkime/aws-aio/issues/116), we should start enable the user to specify the Viewer VPC's CIDR.

Acceptance Criteria

• User able to specify the CIDR for the Viewer Nodes when they call cluster-create.
• If they specify a CIDR, the Viewer Nodes should live in their own VPC.
• If the Viewer CIDR is not specified, then the Viewer Nodes should live in the Capture VPC (i.e. the way it currently works)
• The CLI should reject changes to this CIDR/plan after initial Cluster Creation
• The CLI should reject CIDRs that are "too small" (however that is defined)

[chelma]

Some useful documentation links:

CDK Shapes for TGW

• https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.CfnTransitGateway.html
• https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.CfnTransitGatewayAttachment.html

TGW References

• https://docs.aws.amazon.com/vpc/latest/tgw/tgw-transit-gateways.html
• https://docs.aws.amazon.com/vpc/latest/tgw/tgw-vpc-attachments.html
• https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-share.html
• https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-isolated-shared.html
• https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-quotas.html

[chelma]

Brainstorming how to approach the TGWs, Cfn stacks, etc...

• Some facts about TGWs
  • There's a (soft) limit of 5 TGWs per AWS Account
  • TGWs don't live "inside" a "host VPC" - they are independent. Attachments are associated with VPCs.
  • It's expected that (Num Capture VPCs) <= (Num Viewer VPCs)
  • We might not need to create a TGW for a Capture VPC; depends on whether user specifies a Viewer CIDR
  • If you let them, TGWs will automatically handle routing and propagation, assuming your CIDRs don't conflict
  • You can tell the TGW to accept any attachment requests. If it's within the same account, it should just work. If it's a cross-account request, it requires the TGW account owner grant the attachment account owner access to the TGW using AWS Resource Manager.
• Based on those points, the following approach seems reasonable:
  • Have the TGW be in a separate Cfn stack in the Capture VPC account. This will enable us to optionally create the TGW when needed without messing with the existing Capture Stacks. Additionally, the TGW seems conceptually tied to the Capture VPC than the Viewer VPCs; it exists to the provide access to the "service" that is the Capture VPC.
  • The TGW stack should probably also contain an Attachment for the Capture VPC. For now, it seems fine to do automatic routes/propagation/attachment acceptance. This effectively means that any Viewer VPC in the same AWS Account can easily peer itself with the Capture VPC (and any other Viewer VPCs connected to its TGW). We can optimize routing later so that multiple Viewer VPCs don't expose themselves to eachother via the TGW.
  • We make a new Cfn stack for the Viewer's VPC and its TGW Attachment
  • We update the code to stick the existing Viewer Nodes stack in the new Viewer VPC stack
• Additional considerations
  • This is more a problem for later when we do the Centralized Viewer, but figuring out the transitive routing (e.g. can this new Viewer VPC peer with this Capture TGW) is likely to be tricky. For now, we just need to check if the Capture and Viewer CIDRs overlap.

[chelma]

TODO list looks something like:

• Add CLI Option for cluster-create
• Update cluster-create
  • Update Capacity Plan stuff, including storage/retrieval in Param Store
  • Update checks to ensure we aren't changing the CIDR after creation
  • Update which stacks should be deployed
  • Update CDK Context to have the right values
• Update cluster-destroy
  • Probably need a way to know whether this is a dual-VPC cluster or not (param store?)
  • Update which stacks should be destroyed
  • Update CDK Context to have the right values
• Create the ClusterTGW Stack
  • TGW and attachment to Cluster VPC
• Create the Viewer VPC Stack
  • VPC and attachment to Cluster TGW
• Update CDK Code to deploy the Viewer Nodes in the Viewer VPC if it exists

[chelma]

PR merged; AC met. Resolving.