k3s-io / k3s

Lightweight Kubernetes
https://k3s.io
Apache License 2.0
26.71k stars 2.24k forks source link

Best practice of setting the cluster name at deployment level #10066

Closed fmunteanu closed 3 weeks ago

fmunteanu commented 3 weeks ago

Environmental Info: K3s Version: v1.29.4+k3s1 (94e29e2e)

Node(s) CPU architecture, OS, and Version:

Linux boreas 6.8.0-1004-raspi #4-Ubuntu SMP PREEMPT_DYNAMIC Sat Apr 20 02:29:55 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Cluster Configuration: 3 servers, 5 agents

Describe the bug: My goal is to deploy 2 K3s clusters and set a distinct name for each cluster. Current configuration on both clusters, please note the default name:

# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.4.3:6443
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    client-certificate-data: DATA+OMITTED
    client-key-data: DATA+OMITTED

Ideally, I want to have the clusters named development and production.

K3s generates the /etc/rancher/k3s/k3s.yaml file, where the cluster name is present. I looked at the documentation and there is no setting allowing me to define the cluster name, on initial cluster deployment. How is the default name defined into k3s.yaml file?

I understand the end-user will configure their /.kube/config file the way they want, but I would like to have some sort of --cluster-name switch allowing us to define the cluster name, at deployment level.

Thank you.

brandond commented 3 weeks ago

This is a duplicate of https://github.com/k3s-io/k3s/issues/4610

Clusters don't know their own name, there is no field within the cluster itself to store that data. If you want the kubeconfig to reflect a different name, your best option is to make that edit when collecting the admin kubeconfig off the server nodes, or (preferably) when generating unique kubeconfigs for your users with distinct users and RBAC.

fmunteanu commented 3 weeks ago

Thank you @brandond for providing the clarifications. I was simply trying to understand how the default value is assigned when the cluster is deployed and if there is a way to define it with a switch, during deployment. I still believe this could be an useful feature.

brandond commented 3 weeks ago

We've discussed it in the past and decided against it. The admin kubeconfig really shouldn't be used for much other that locally interacting with the server. If you're managing multiple clusters to the point where you want to give their contexts unique names in your kubeconfig, you should be using more advanced tooling for that - and not using the default admin kubeconfig and admin RBAC.

fmunteanu commented 3 weeks ago

Thank you, it makes sense now.