neolit123
commented
3 years ago This is a feature request that kubeadm certs generate-csr (or a new similar command) should support worker nodes. Currently this command seems to be built with the assumption that it's being executed on a control plane node.
yes, basically kubeadm certificates are managed on the init node and that's what certificate / CSR commands are targetting. the UX on joining nodes is oriented towards using a simple configuration / keys / tokens. this is by design and it satisfies the majority of use cases.
Worker node boots with a kubeadm config file containing just the JoinConfiguration and KubeletConfiguration (I believe these are all that we should need for a worker node)
note, currently we don't support KubeletConfiguration on join but we might be adding it soon.
https://github.com/kubernetes/kubeadm/issues/1682
The workflow I would like to achieve is the following:
can you explain why you need a CSR for the kubelet client certificate? (EDIT: ok i saw that you just want consistency with the rest of your setup)
i think you can simplify this if you have a look at our e2e test for external CA: https://github.com/kubernetes/kubeadm/blob/05f9b96cf83d73ed2177815487eba90517b66708/kinder/pkg/cluster/manager/actions/setup-external-ca.go#L28 https://github.com/kubernetes/kubeadm/blob/master/kinder/ci/workflows/external-ca-tasks.yaml
what happens is:
- you can generate a kubelet.conf directly on the joining nodes using:
kubeadm init phase kubeconfig kubelet --control-plane-endpoint=%s(initdoes require InitConfiguration / ClusterConfiguration or you can use some of the flags like --control-plane-endpoint) or generate them externally but pass InitConfiguration.NodeRegistrationOptions.Name to be explicit about the node name in the cert. - then for join you can do
kubeadm join ... --ignore-preflight-errors=FileAvailable--etc-kubernetes-kubelet.conf,FileAvailable--etc-kubernetes-pki-ca.crtthis will make kubeadm ignore the fact that there is akubelet.conf/ca.crtalready. - the bootstrap will complete using this
kubelet.confand you don't have to execute only thekubeadm join phase kubelet-startphase.
I can't run kubeadm certs generate-csr without the kubeadm config file having an InitConfiguration or ClusterConfiguration. These appear to be control plane node configuration files to me, so they shouldn't need to be included on worker nodes just to be able to generate CSRs.
this is by design as mentioned above.
I end up with CSRs for all of the control plane components. Not a problem per se, just a bit unnecessary so it would be nice to be able to generate the CSR just for kubelet.conf.
this is something that we can optimize to be similar to the certs command which allows asking for specific certs.
I need to manually override the server in kubelet.conf to point it at the API server address for the control plane since by default the the kubeconfig files contain the IP of the local node. This makes sense for control plane nodes so that components only connect to the local API server, but obviously it doesn't work for a worker node where the API server isn't running.
in our external CA e2e we just do kubeadm init phase kubeconfig kubelet --control-plane-endpoint=%s, but we don't generate via CSRs. i think generate csr would allow you to configure this using the ClusterConfiguration?
Please let me know if you think there are any better ways of accomplishing the same goals. I'm also happy to do the implementation if this idea gains any support. It's probably only worth doing if you think it would be useful for anyone else since I already have a workaround.
given you are the first user demanding such a change i would hold until there is more demand. and given there are a number of workarounds to support external CA / joining without tokens.
robbiemcmichael
gopaltirupur
Summary
This is a summary of the feature request after some design discussion in the comments below.
External CA mode could be better supported for worker nodes by changing the
generate-csrcommand to support the following modes:The name for the sub-command could be either
kubeletorkubelet.conf. To me it seems thatkubeletis more consistent with the existingkubeadm init phase kubeconfig kubeletcommand.Changing the original
generate-csrcommand to require theallsub-command would be a breaking change. This is on hold unless there is further interest from others in this feature.Original request
This is a feature request that
kubeadm certs generate-csr(or a new similar command) should support worker nodes. Currently this command seems to be built with the assumption that it's being executed on a control plane node.Goal
I'm trying to implement a declarative join process for worker nodes when using an external CA. By this I mean that I cannot sensibly generate a bootstrap token from the control plane nodes and get it to the worker node, so instead I'm using file based discovery with an out-of-band process for signing certificates.
The workflow I would like to achieve is the following:
JoinConfigurationandKubeletConfiguration(I believe these are all that we should need for a worker node)kubeadm certs generate-csris used to generate the private key and CSR forkubelet.confkubelet.confto embed the private key and certificate after signing it with the external CA (e.g. this can be done with the Vault PKI API)kubeadm join phase kubelet-startwhich uses file based discovery to connect to the Kubernetes API and fetchClusterConfigurationfrom a configmapProblem
This very nearly works already, I've just hit a few hiccups:
kubeadm certs generate-csrwithout the kubeadm config file having anInitConfigurationorClusterConfiguration. These appear to be control plane node configuration files to me, so they shouldn't need to be included on worker nodes just to be able to generate CSRs.kubelet.conf.kubelet.confto point it at the API server address for the control plane since by default the the kubeconfig files contain the IP of the local node. This makes sense for control plane nodes so that components only connect to the local API server, but obviously it doesn't work for a worker node where the API server isn't running.Right now it's easier easier just to avoid generating the CSR and manually generate the certificate with the correct fields, but I would like to be able to generate the CSR for a worker node so I can remove this code and use the same process that I use for control plane nodes.
To address the issues above, I'd like to add a way to generate the CSR which doesn't require any control plane configuration files, only generates the CSR for
kubelet.confand generates thekubelet.confwith the.clusters[0].cluster.serverfield pointing to the control plane API server.Please let me know if you think there are any better ways of accomplishing the same goals. I'm also happy to do the implementation if this idea gains any support. It's probably only worth doing if you think it would be useful for anyone else since I already have a workaround.