Separate subnets for VPC endpoints

alex-dabija commented 1 year ago

Story

-As a cluster admin, I want the VPC endpoints to use separate subnets (one per availability zone) in order to have clear network boundaries.

Towards epic.

Background

Giantswarm's implementation of CAPA creates a few VPC endpoints which use the same subnets the cluster nodes.

Some customers want a clear separation between the cluster sub-components because they use a transit gateway which filters traffic based on its source (CIDR).

The aws-vpc-operator is able to create all the required subnets with CIDRs configured.

Requirements

separate subnets for the VPC endpoints;
each subnet CIDR needs to be configurable;

TODOs

[ ] expose configuration settings via values.yaml in cluster-aws to configure the bastion hosts subnets.

freudl commented 1 year ago

Should we ensure that AWS Secrets Manager Private Endpoint exists and is accessible via TGW so you don't need to provision this?

alex-dabija commented 1 year ago

Implemented and released as part of cluster-aws v0.21.0.

alex-dabija commented 1 year ago

Should we ensure that AWS Secrets Manager Private Endpoint exists and is accessible via TGW so you don't need to provision this?

@freudl I quickly looked into this today and there are 2 AWS resources with the same domain name (secretsmanager.eu-central-1.amazonaws.com): the VPC endpoint we created and the resolver rule shared by NOC. I don't know which takes precedence.

Looking at the facts, we'll probably have to disable the VPC Endpoint creation.

alex-dabija commented 1 year ago

Implemented and tested on thor.

giantswarm / roadmap