Brainstorming how to approach the TGWs, Cfn stacks, etc...
Some facts about TGWs
There's a (soft) limit of 5 TGWs per AWS Account
TGWs don't live "inside" a "host VPC" - they are independent. Attachments are associated with VPCs.
It's expected that (Num Capture VPCs) <= (Num Viewer VPCs)
We might not need to create a TGW for a Capture VPC; depends on whether user specifies a Viewer CIDR
If you let them, TGWs will automatically handle routing and propagation, assuming your CIDRs don't conflict
You can tell the TGW to accept any attachment requests. If it's within the same account, it should just work. If it's a cross-account request, it requires the TGW account owner grant the attachment account owner access to the TGW using AWS Resource Manager.
Based on those points, the following approach seems reasonable:
Have the TGW be in a separate Cfn stack in the Capture VPC account. This will enable us to optionally create the TGW when needed without messing with the existing Capture Stacks. Additionally, the TGW seems conceptually tied to the Capture VPC than the Viewer VPCs; it exists to the provide access to the "service" that is the Capture VPC.
The TGW stack should probably also contain an Attachment for the Capture VPC. For now, it seems fine to do automatic routes/propagation/attachment acceptance. This effectively means that any Viewer VPC in the same AWS Account can easily peer itself with the Capture VPC (and any other Viewer VPCs connected to its TGW). We can optimize routing later so that multiple Viewer VPCs don't expose themselves to eachother via the TGW.
We make a new Cfn stack for the Viewer's VPC and its TGW Attachment
We update the code to stick the existing Viewer Nodes stack in the new Viewer VPC stack
Additional considerations
This is more a problem for later when we do the Centralized Viewer, but figuring out the transitive routing (e.g. can this new Viewer VPC peer with this Capture TGW) is likely to be tricky. For now, we just need to check if the Capture and Viewer CIDRs overlap.
Description
As part of implementing (https://github.com/arkime/aws-aio/issues/116), we should start enable the user to specify the Viewer VPC's CIDR.
Acceptance Criteria
cluster-create
.