Error from server: error dialing backend: x509: certificate signed by unknown authority

[tsgan]

When trying to run command in one pod on worker-2 gives:

Error from server: error dialing backend: x509: certificate signed by unknown authority

it works for worker-1.

NAMESPACE     NAME                       READY   STATUS    RESTARTS   AGE   IP             NODE       NOMINATED NODE   READINESS GATES
default       busybox                    1/1     Running   0          20s   10.44.0.3      worker-1   <none>           <none>
default       busybox2                   1/1     Running   1          91m   10.32.0.4      worker-2   <none>           <none>
default       nginx-6799fc88d8-kvxv6     1/1     Running   1          87m   10.44.0.2      worker-1   <none>           <none>
kube-system   coredns-5cccc7b6cc-hfsmc   1/1     Running   5          18h   10.32.0.2      worker-2   <none>           <none>
kube-system   coredns-5cccc7b6cc-qcrdb   1/1     Running   8          18h   10.44.0.1      worker-1   <none>           <none>
kube-system   weave-net-xsrd2            2/2     Running   8          19h   192.168.5.22   worker-2   <none>           <none>
kube-system   weave-net-znb9v            2/2     Running   1          44m   192.168.5.21   worker-1   <none>           <none>
root@master-1:~/test-infra# kubectl exec -ti busybox -- nslookup kubernetes
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

root@master-1:~/test-infra# kubectl exec -ti busybox2 -- nslookup kubernetes
Error from server: error dialing backend: x509: certificate signed by unknown authority

[vCillusion]

I faced the same issue for worker-2, getting permissions issue for worker-1 though.

[vCillusion]

I faced the same issue for worker-2, getting permissions issue for worker-1 though.

able to work worker-1 with correct user name for kube-apiserver instead of system:kube-apiserver

[JasonYLong]

the Author has changed the document on 13-kube-apiserver-to-kubelet.md . changed part from "name: system:kube-apiserver" to "name: kube-apiserver" . It can fixed this issue

cat <<EOF | kubectl apply --kubeconfig admin.kubeconfig -f - apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: system:kube-apiserver namespace: "" roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:kube-apiserver-to-kubelet subjects:

• apiGroup: rbac.authorization.k8s.io kind: User name: kube-apiserver EOF

[marioerceg]

Faced the same issue. Indeed, not received server CSR, only client which was auto-approved. Resolved by adding --rotate-server-certificates=true \ to /etc/systemd/system/kubelet.service file and restarting service.

`cat <<EOF | sudo tee /etc/systemd/system/kubelet.service [Unit] Description=Kubernetes Kubelet Documentation=https://github.com/kubernetes/kubernetes After=docker.service Requires=docker.service

[Service] ExecStart=/usr/local/bin/kubelet \ --bootstrap-kubeconfig="/var/lib/kubelet/bootstrap-kubeconfig" \ --config=/var/lib/kubelet/kubelet-config.yaml \ --image-pull-progress-deadline=2m \ --kubeconfig=/var/lib/kubelet/kubeconfig \ --cert-dir=/var/lib/kubelet/pki/ \ --rotate-certificates=true \ --rotate-server-certificates=true \ --network-plugin=cni \ --register-node=true \ --v=2 Restart=on-failure RestartSec=5

[Install] WantedBy=multi-user.target EOF`

and then approve CSR on the master node. Tutorial should be updated with this

[fitszhang]

It worked after change the service configuration from @marioerceg Thanks

[ivarabyeu]

@tsgan did you solve the problem?

[JasonYLong]

the Author has changed the document on 13-kube-apiserver-to-kubelet.md . changed part from "name: system:kube-apiserver" to "name: kube-apiserver" . It can fixed this issue .

[fireflycons]

Hi @tsgan @vCillusion @fitszhang @iyuvorobyov @JasonYLong

We have now merged a major change to bring this to v1.24. This has been addressed as part of the upgrade. Please try it now and feel free to raise further issues.

Thanks.