search

xetys / hetzner-kube

A CLI tool for provisioning kubernetes clusters on Hetzner Cloud
Apache License 2.0
745 stars 116 forks source link

[Idea] Support multi-zone K8S Cluster #253

Open md2k opened 5 years ago

md2k commented 5 years ago

Not sure if it comes already or not, but maybe it is good idea to think about multi-zone K8S cluster configuration where we can roll control-plane and workers specifying 2-3 different DCs what can increase high availability and redundancy ?

xetys commented 5 years ago

AFAIK we do 2 things on this:

  1. per default we use at least 2 DCs to provision hetzner kube nodes. In this case, the DCs are picked in Round Robin style, so both, workers and controller nodes are spreaded there. This means that the control plane in a HA setup are at least in 2 DCs by default
  2. you may choose to install the hcloud-controller-manager addon, which automatically adds failure domains on these node, which k8s can use for failover. Here I am not sure, if this addon still works with the current version. This should be checked.
md2k commented 5 years ago

Hm, maybe it because i'm explicitly set DC to deploy.... my bad :)

hcloud-controller-manager when i tried to deploy it - it has failed. not sure already what problem was, i'll check it again. if i not wrong, it was something about flannel, but not sure what exactly.

md2k commented 5 years ago

yeah.. here another problem with multi-zone... floating IP bounded to DC... so it will be complicated to balance traffic and can create complications for ingress controllers :( unless we can add additional parameter to enable floating IP per master or per worker (optional) to have static addresses which we can then use for dns and so on. but yeah... this part with hetzner not easy at this moment. can see only some kind of 2-3 instance pool rolled with terraform and traefik inside as traffic balancers and pointing statically to all workers. not sure what to do with this puzzle. If everyone in single DC all ok , it is possible to use keepalived with floating IP. But with multi-DC out from ideas :( (only costly DynNect Traffic Director or AWS Traffic Flow as DNS balancers and end-point health checks)