search

jhunt / k8s-boshrelease

A BOSH Release for deploying Kubernetes clusters
MIT License
13 stars 9 forks source link

Upgrade to etcd 3.3.14 (minimum) to fix gRPC etcd security issues #24

Closed jhunt closed 4 years ago

jhunt commented 4 years ago

v3.3.14 fixed some stuff that affects Kubernetes use cases:

v3.3.14 had to include some features from 3.4, while trying to minimize the difference between client balancer implementation. This release fixes "kube-apiserver 1.13.x refuses to work when first etcd-server is not available" (kubernetes#72102).

(from https://github.com/etcd-io/etcd/releases/tag/v3.3.14)

This is in reference to https://github.com/kubernetes/kubernetes/issues/72102, which I am seeing on one of our larger k8s installations.

My API node logs are filled with these errors:

W1111 19:32:12.443121    3194 clientconn.go:1120] grpc: addrConn.createTransport failed to connect to {https://10.128.4.18:2379 0  <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate is valid for 127.0.0.1, 10.128.4.18, not 10.128.4.17". Reconnecting...
W1111 19:32:12.444838    3194 clientconn.go:1120] grpc: addrConn.createTransport failed to connect to {https://10.128.4.19:2379 0  <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate is valid for 127.0.0.1, 10.128.4.19, not 10.128.4.17". Reconnecting...
W1111 19:32:12.445557    3194 clientconn.go:1120] grpc: addrConn.createTransport failed to connect to {https://10.128.4.19:2379 0  <nil>}. Err :connection error: desc = "transport: authentication handshake failed: x509: certificate is valid for 127.0.0.1, 10.128.4.19, not 10.128.4.17". Reconnecting...

Upgrade to 3.3.14 and test it out.