search

k3s-io / k3s

Lightweight Kubernetes
https://k3s.io
Apache License 2.0
26.62k stars 2.24k forks source link

Use private ip with external cloud manager that doesn't know about that ip #10032

Closed gsimko closed 2 weeks ago

gsimko commented 2 weeks ago

Is your feature request related to a problem? Please describe.

I'm trying to set up wireguard to protect all the communications between my k8s nodes. I want that to include the initialization steps as well (the script from https://get.k3s.io), so the flannel CNI cannot address this. I'm doing this in the hetzner cloud, where I cannot use the hetzner private network because it doesn't support connections between different regions and I'm aiming for global coverage.

The problem is that if I set node-ip to the desired private ip, the CCM fails because it won't be able to identify the node by the "alpha.kubernetes.io/provided-node-ip" label. Furthermore, this label is not allowed to be initialized on the kubelet startup, so I cannot change it.

Describe the solution you'd like It would be ideal to separate the two uses of node-ip and add control to manage them separately:

  1. it controls where the kube-apiserver and etcd listens (this is what I'd like to set to my private ip)
  2. it controls how to CCM matches the node through "alpha.kubernetes.io/provided-node-ip" (this is where I'd like to keep using the original public ip)

I'm also open for alternative suggestions, but I couldn't figure out any.