search

mbruzek / layer-k8s

A repository for the Kubernetes charm implmeneted in the reactive framework.
9 stars 3 forks source link

TLS Certificate is missing SDN address for Kubernetes API #44

Closed lazypower closed 8 years ago

lazypower commented 8 years ago

In our default K8s setup, the TLS certificate is generated and signed for serving on the units public-address, and private-address. However its not uncommon for services to want to connect to kubernetes using the SDN internal address it broadcasts to the cluster.

Unless we are going to update the K8s master to broadcast its address as the private address, we should add the SDN ip address assigned to the service. (10.1.0.1 in the default CIDR setup, 99% of the time)

This address (range) can be changed with the CIDR setting of flannel, which makes this tricky.

mbruzek commented 8 years ago

This is a good catch @chuckbutler. It may be something we need to change in the layer-tls so other charms can provide additional Subject Alternative Names (SANs) for their certificates.

The get_sans() method accepts a list of additional addresses but the layer-tls does not pass in any additional addresses: https://github.com/juju-solutions/layer-tls/blob/master/reactive/tls.py#L283

What would be a well designed generic way to pass in this information so the other charms that use the layer-tls could do the same?

lazypower commented 8 years ago

Ideally all certificates are generated from CSR's, and we can make those CSR's request every IP the unit knows about. Including those that come from config or from the IAAS provider, or an SDN module. Which would then get passed appropriately to the get_sans() method from that CSR breakdown.

We did a "leader has special circumstances" in the current code. The certificate on the leader gets generated early on implicitly, which in itself is troublesome. I think to pass more information in we need to make this an event, or an explicit invocation - but not necessarily triggered by the tls layer. But that is not great either, every consumer has to explicitly set a state or something to trigger the cert to get generated.

I'm pretty sure we are hard-coding the 10.1.0.1 in kubernetes. What if we amended the implied SANS with the IP of flannels interface? Giving SDN providers a chance to take part in the TLS CSR generation?

The idea is the SDN provider knows what device, ip mapping, etc. It makes sense to interrogate that module for the information.

mbruzek commented 8 years ago

We encountered this bug today: 2016/07/06 14:45:16 Readinesscheck error listing namespaces (Get https://10.1.0.1:443/api/v1/namespaces: x509: certificate is valid for 54.153.72.46, 172.31.21.255, not 10.1.0.1)