search

habitat-sh / core-plans

Core Habitat Plan definitions
130 stars 252 forks source link

[kubernetes-controller-manager][kubernetes-scheduler] Enable HA K8s clusters #1534

Open HT154 opened 6 years ago

HT154 commented 6 years ago

It's almost possible to use the core/kubernetes-* packages to run an HA Kubernetes cluster.

The apiserver, kubelet, and proxy packages already run in the correct topology for an HA cluster.

The controller-manager and scheduler components must run once per cluster, but this is not supported in the current plans. For an HA cluster, these services should support the leader topology—if the member is elected leader, the service will start normally; if not elected leader, the run hook should sleep.

HT154 commented 6 years ago

Note: this is functionality that Kubernetes provides itself! See the --leader-elect flag here: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/

This functionality relies on the k8s' etcd datastore as a distributed lock. I'd rather reduce the datastore traffic and let Habitat handle the application topology in its entirely.