Overhaul Kubernetes integration

Quentin-M / etcd-cloud-operator

Deploying and managing production-grade etcd clusters on cloud providers: failure recovery, disaster recovery, backups and resizing.

Apache License 2.0

234 stars 42 forks source link

While the etcd-cloud-operator has been serving us amazingly on top of AWS the past several years, support for Kubernetes within the etcd-cloud-operator has been 3rd party contributed and not directly supported / tested. This PR provides a first pass of fixes.

CHANGELOG

Removes dependency on the Kubernetes API for discovery
Fixes use of PVCs on Kubernetes, as well as snapshot restores on Kubernetes

TODO

A lot more (DR) testing, network/node gone, etc
Revamp of the manifests, docs, enabling Client TLS by default
Idea: optionally bring on a lightweight monitoring stack pre-configured with a dashboard and alerts (I got that handy).

QUICK TESTING

helm template --namespace "dev" --release-name "test" etcd-cloud-operator | kubectl apply -n dev -f -

I am having an issue starting this version of etcd on our dev cluster due to the following error, which is addressed quite nicely in 3.5 by just exposing SO_REUSEPORT, etc. I had to comment out the metrics server, should dive deeper into the issue and/or make metrics configurable if needed.

In the meantime, I think we can merge as this provides value to local testing and Kubernetes usage, which nobody I believe uses and therefore does not require backward compatibility.

level=error msg="failed to seed the cluster" error="failed to start etcd: listen tcp 127.0.0.1:2381: bind: address already in use"

//etcdCfg.ListenMetricsUrls = append(metricsURLs(c.cfg.BindAddress), metricsURLs("127.0.0.1")...)
//etcdCfg.Metrics = "extensive"

Quentin-M / etcd-cloud-operator