search

siderolabs / omni-feedback

Omni feature requests, bug reports
https://www.siderolabs.com/platform/saas-for-kubernetes/
MIT License
2 stars 0 forks source link

[feature] predictable controlplane endpoints #39

Closed mcflis closed 1 year ago

mcflis commented 1 year ago

Problem Description

When creating a cluster, the cluster endpoint is picked by the omni system automatically (see https://github.com/siderolabs/omni-feedback/issues/22#issuecomment-1505126598).

Theoretically, this makes it impossible to install Cilium CNI without kube-proxy enabled, because one needs to know the endpoint in advance according to https://www.talos.dev/v1.4/kubernetes-guides/network/deploying-cilium/

However, my observations show that the same endpoint address (unique local ipv6) is picked when a cluster is recycled (i.e., delete the cluster and recreate it). This holds true for one cluster tied to an Omni account, but it is not clear how stable this approach is or even if it is stable at all.

Solution

Allow to manage controlplane endpoints from Omni UI and omnictl and assign them (like machines) to a cluster. When adding an endpoint, the endpoint address is selected automatically, but can be patched like MachineConfig, etc.

This would help manage the custom/local endpoints I've suggested in #22 where one could add an endpoint and override the endpoint address using a patch.

Alternative Solutions

Show the next endpoint address in the Omni home screen:

image

Notes

No response

smira commented 1 year ago

You don't really the Kubernetes endpoint before the cluster is created. You can create the cluster as usual, pull down kubeconfig, and install Cilium using either CLI or Helm once the Kubernetes API server is up.

smira commented 1 year ago

You don't really the Kubernetes endpoint before the cluster is created. You can create the cluster as usual, pull down kubeconfig, and install Cilium using either CLI or Helm once the Kubernetes API server is up.

Correction: you can find the endpoint in the machine config of any node.

mcflis commented 1 year ago

Mhm, I see, but this makes automation hard since I have to create a cluster to see the endpoint, then update the respective files in my IaC git repo (and maybe recreate the cluster).

However, I'll close this issue because it's redundant and you are already looking into #22.