kubernetes-sigs / cluster-api-provider-aws

Kubernetes Cluster API Provider AWS provides consistent deployment and day 2 operations of "self-managed" and EKS Kubernetes clusters on AWS.
http://cluster-api-aws.sigs.k8s.io/
Apache License 2.0
627 stars 548 forks source link

Subnet division enhacement #4784

Open MaxFedotov opened 5 months ago

MaxFedotov commented 5 months ago

/kind feature

Describe the solution you'd like By default, Cluster API AWS organizes subnets so that most of the VPC address space goes to private subnets. While this is true for most classic Kubernetes deployments, that may not work in some specific cases.

We in Wargaming use Kuberentes to schedule arenas with gameservers using google agones. These arenas requires to be run on a node with a public IP address using hostPort. As pods with arenas are the main cluster workload, we need to dedicate most of VPC address space to the public subnets, instead of private.

In order to support this requirement, I would like to propose an additional configuration parameter in AWSCluster.spec.network.vpc - SubnetSchema with the following enum values:

richardcase commented 4 months ago

This seems reasonable to me. I'd favor the simplest solution to meet this (i.e. the same percentages but flipped).

I think anything more complex (like changing percentages) should be left for the network re-work (i.e. networking v2) in #1484.

MaxFedotov commented 4 months ago

I love the idea of percentages, but the exact realization seems very complicated to me due to the following conditions:

So if a user specifies something like "I want 37% of my address space to be allocated to public subnets and I also want to have them spread in 3 AZ", it will be impossible to split them correctly using the provided percentage without performing some round-up or other approximations.

richardcase commented 4 months ago

/triage accepted

dlipovetsky commented 4 months ago

Subnets are divided into public and private here.

I'm ok with a simple API that supports two choices (mostly public, or mostly private), but I think we need to clearly document that, for any other choices, users must define their own subnets (although https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/1484 could change that).

MaxFedotov commented 4 months ago

Ok, let's then start with two options: PreferPrivate (the current one and which will be selected by default if nothing is specified) and PreferPublic