search

replicatedhq / kots

KOTS provides the framework, tools and integrations that enable the delivery and management of 3rd-party Kubernetes applications, a.k.a. Kubernetes Off-The-Shelf (KOTS) Software.
https://kots.io
Apache License 2.0
901 stars 90 forks source link

Allows kots work on Kind by changing Node affinity criteria #3361

Open camilamacedo86 opened 2 years ago

camilamacedo86 commented 2 years ago

Description

Kots has some NodeAffinity criteria implementations which require the label node-role.kubernetes.io/masterwhich does not necessarily exist for all K8s vendors. Therefore, this issue is to propose a suggestion for replacing RequiredDuringSchedulingIgnoredDuringExecution with preferredDuringSchedulingIgnoredDuringExecution so that it still be able to be scheduled with/when is not possible to match but ensure its preference which would allow work on more vendors by default. You can check the k8s doc for further info: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity.

See the code spot: https://github.com/replicatedhq/kots/blob/c597a4aebbf5caead94419b860c3d2311146d702/pkg/kurl/join_cert.go#L208-L229

Motivation

Note that Kind for example will not have this label: (Following an example with a default basic config)

(i.e.) Kind cluster with 3 nodes created with the following config (kind v0.16.0 / k8s 1.25.2)

$ cat kind-example-config.yaml 
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
kind create cluster --config=kind-example-config.yaml

See that we have not node-role.kubernetes.io/master label

$ kind get nodes 
kind-worker2
kind-control-plane
kind-worker
 $ kubectl label node kind-control-plane --list
kubernetes.io/hostname=kind-control-plane
kubernetes.io/os=linux
node-role.kubernetes.io/control-plane=
node.kubernetes.io/exclude-from-external-load-balancers=
beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64

$ kubectl label node kind-worker2 --list
beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
kubernetes.io/arch=arm64
kubernetes.io/hostname=kind-worker2
kubernetes.io/os=linux

$ kubectl label node kind-worker --list
kubernetes.io/arch=arm64
kubernetes.io/hostname=kind-worker
kubernetes.io/os=linux
beta.kubernetes.io/arch=arm64
beta.kubernetes.io/os=linux
adamancini commented 1 year ago

Looks like the original master label is being deprecated https://kubernetes.io/docs/reference/labels-annotations-taints/#node-role-kubernetes-io-master-taint