equinix-labs / terraform-equinix-metal-eks-anywhere

EKS Anywhere on Equinix Metal (Baremetal)
https://deploy.equinix.com/labs/terraform-equinix-metal-eks-anywhere/
Apache License 2.0
9 stars 5 forks source link

Consider use of private ranges within the VLAN #4

Open displague opened 2 years ago

displague commented 2 years ago

Within the VLAN, each node could benefit from a private network range for cluster addressing. Today each node is only assigned a public IP address from a public IP reservation pool.

Note: The private address ranges would not need to be assigned to the gateway. Any arbitrary range could be used.

Instructing the internal nodes to bind to this address will require support for that capability in EKS-A on Baremetal, wether through new hardware.csv fields or other means of overriding the netplan, cloud-config, and or hegel metadata.

displague commented 2 years ago

Alternatively, create a NAT on the EKS-A Admin node and assign private addresses to the VLAN nodes. The Metal GW would become optional and public addressing would need to be provided through a tool like kube-vip or metallb.

displague commented 2 years ago

Kube-vip support is available: https://anywhere.eks.amazonaws.com/docs/tasks/workload/loadbalance/kubevip/ This could allow for tighter integration with Equinix Metal features: https://metal.equinix.com/developers/guides/kube-vip-type-lb/

displague commented 2 years ago

Some users have reported difficulty configuring or using hybrid-bonded + Gateway. For those cases, the NAT approach could be attempted with hybrid-unbonded.

FWIW, we haven't seen difficulties in using the hybrid-bonded + Gateway approach aside from systemctl restart networking not taking in some cases. https://github.com/equinix-labs/terraform-equinix-metal-eks-anywhere/issues/9

displague commented 1 year ago

Instructing the internal nodes to bind to this address will require support for that capability in EKS-A on Baremetal, wether through new hardware.csv fields or other means of overriding the netplan, cloud-config, and or hegel metadata.

The EKS-A Tinkerbell integration only supports a single-node address today.