equinix-labs / terraform-equinix-metal-k3s

Manage K3s (k3s.io) region clusters on Equinix Metal
https://registry.terraform.io/modules/equinix/k3s/metal/latest?tab=readme
Apache License 2.0
46 stars 15 forks source link

Use Private IPs and backend transfer for internal node addresses #80

Open displague opened 6 months ago

displague commented 6 months ago

The nodes in this cluster use their public IP addresses for intra-cluster communication.

root@k3s-demo-cluster-k3s-cp-aio:~# kubectl get nodes -o wide
NAME                          STATUS   ROLES                  AGE     VERSION        INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION    CONTAINER-RUNTIME
k3s-demo-cluster-k3s-cp-aio   Ready    control-plane,master   5m41s   v1.28.7+k3s1   145.xx.xx.xx   <none>        Debian GNU/Linux 11 (bullseye)   5.10.0-28-amd64   containerd://1.7.11-k3s2

For demo purposes, with more consideration of best practices and security concerns, these POC nodes should be deployed with public and private addresses. Project Backend Transfer should be used to ensure the cluster nodes can communicate across metros.

The public addresses could then be used for exposing services on public node ips.

e-minguez commented 6 hours ago

@displague PTAL at the #87 PR.

I'm not sure about enabling project backend transfer, let me explain.

Currently clusters are metro isolated (you cannot deploy a cluster with nodes on different metros) hence it shouldn't be any kind of communication between clusters. If there is any kind of traffic between cluster it should be routed/managed as regular traffic as I don't think there is a way to specify different "external ips" depending on the source.

WDYT?