canonical / microk8s

MicroK8s is a small, fast, single-package Kubernetes for datacenters and the edge.
https://microk8s.io
Apache License 2.0
8.43k stars 770 forks source link

Windows node invalid certificate (Certificate does not include any IP SANs) #4522

Open peer-qvannatter opened 4 months ago

peer-qvannatter commented 4 months ago

https://github.com/canonical/microk8s/blob/66176f295cb76048310ab7d7cadc4a53ccb16a23/microk8s-resources/default-args/kube-apiserver#L30

By default, with the installation of a Windows node a self-signed kubelet.crt is created.

Currently, there's two issues with the certificate that's created.

  1. It does not include any IP SANs
  2. Because it's self-signed, the certificate authority is not trusted.

This breaks functionality related to "kubectl exec" and "kubectl logs" when accessing resources on a Windows node.

The introduction of "--kubelet-certificate-authority=${SNAP_DATA}/certs/ca.crt" flag by default enables the validation of the certificates which causes all of these new issues.

As a workaround, either the certificate on the windows node needs to be created manually and signed by the control plane certificate or the line "--kubelet-certificate-authority=${SNAP_DATA}/certs/ca.crt" can be removed from "/var/snap/microk8s/current/args/kube-apiserver"

neoaggelos commented 4 months ago

Hi @peer-qvannatter. Indeed, in MicroK8s 1.28 we applied more hardened defaults, one of which was --kubelet-certificate-authority (see https://microk8s.io/docs/how-to-cis-harden#check-125-29)

It would be useful if the Windows worker nodes was updated to include instructions for generating that certificate for worker nodes, such that kubelet proxy calls are trusted.

ncat-epic commented 4 months ago

Hello - just added a Windows worker node and ran into this issue.

What are the instructions to update the Windows worker node certificate and sign it by the control plane CA?

Followed this to add Windows worker node: https://microk8s.io/docs/add-a-windows-worker-node-to-microk8s

peer-qvannatter commented 4 months ago

Assuming you have OpenSSL, these are the steps to create the certificate manually:

  1. Create an CSR with the proper SANs
  1. Copy the CA certificate and key from /var/snap/microk8s/current/certs to [openssl root]/cacert.pem and [openssl root]/private/cakey.pem respectively

  2. Sign the certificate: (You may also need to update your openssl config so it doesn't complain about missing fields when you attempt to sign)

    • sudo sed -i 's/required/optional/g' /usr/lib/ssl/openssl.cnf
    • openssl ca -in kubelet.csr -out kubelet.crt
  3. Copy kubelet.crt and kubelet.key to C:\var\lib\kubelet\pki on the windows node