search

kubernetes / kubeadm

Aggregator for issues filed against kubeadm
Apache License 2.0
3.76k stars 715 forks source link

Kubeadm upgrade plan failed v1.17.11 to 1.18.20 #2560

Closed prabhinmp closed 3 years ago

prabhinmp commented 3 years ago

Hi Team, I'm upgrading Kubernetes cluster from v1.17.11 to 1.18.20.

Environment
master - 1 node
worker- 2 nodes

Im not using external etcd . but I'm getting below error when I do kubeadm upgrade plan. command

I0825 00:49:59.791965 3510173 round_trippers.go:444] GET https://192.168.190.78:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 1 milliseconds I0825 00:49:59.791980 3510173 round_trippers.go:450] Response Headers: I0825 00:49:59.791985 3510173 round_trippers.go:453] Cache-Control: no-cache, private I0825 00:49:59.791991 3510173 round_trippers.go:453] Content-Type: application/json I0825 00:49:59.791998 3510173 round_trippers.go:453] Content-Length: 979 I0825 00:49:59.792011 3510173 round_trippers.go:453] Date: Wed, 25 Aug 2021 04:49:59 GMT I0825 00:49:59.792038 3510173 request.go:1068] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"kubeadm-config","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/kubeadm-config","uid":"de91e9af-15a6-48d6-bd75-c02df7257ed3","resourceVersion":"56870577","creationTimestamp":"2020-10-16T07:26:37Z"},"data":{"ClusterConfiguration":"apiServer:\n extraArgs:\n authorization-mode: Node,RBAC\n timeoutForControlPlane: 4m0s\napiVersion: kubeadm.k8s.io/v1beta2\ncertificatesDir: /etc/kubernetes/pki\nclusterName: kubernetes\ncontrollerManager: {}\ndns:\n type: CoreDNS\netcd:\n local:\n dataDir: /var/lib/etcd\nimageRepository: k8s.gcr.io\nkind: ClusterConfiguration\nkubernetesVersion: v1.17.17\nnetworking:\n dnsDomain: cluster.local\n podSubnet: 10.244.0.0/16\n serviceSubnet: 10.96.0.0/12\nscheduler: {}\n","ClusterStatus":"apiEndpoints:\n kubernetes-master:\n advertiseAddress: 192.168.190.78\n bindPort: 6443\napiVersion: kubeadm.k8s.io/v1beta2\nkind: ClusterStatus\n"}} I0825 00:49:59.792451 3510173 etcd.go:103] etcd endpoints read from pods: https://192.168.190.78:2379 context deadline exceeded error syncing endpoints with etc k8s.io/kubernetes/cmd/kubeadm/app/util/etcd.NewFromCluster /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/util/etcd/etcd.go:119 k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.runPlan /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/plan.go:87 k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.NewCmdPlan.func1 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/plan.go:55 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:826 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:914 k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:864 k8s.io/kubernetes/cmd/kubeadm/app.Run /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:50 main.main _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25 runtime.main /usr/local/go/src/runtime/proc.go:203 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1357

Kindly help on this.

prabhinmp commented 3 years ago

found below log in etcd during upgrade plan command

021-08-25 05:54:34.838997 I | embed: rejected connection from "192.168.190.78:39392" (error "remote error: tls: bad certificate", ServerName "")
2021-08-25 05:54:36.617835 I | embed: rejected connection from "192.168.190.78:39396" (error "remote error: tls: bad certificate", ServerName "")
2021-08-25 05:54:39.424431 I | embed: rejected connection from "192.168.190.78:39400" (error "remote error: tls: bad certificate", ServerName "")
2021-08-25 05:54:42.805490 I | embed: rejected connection from "192.168.190.78:39406" (error "remote error: tls: bad certificate", ServerName "")
2021-08-25 05:54:49.898988 I | embed: rejected connection from "192.168.190.78:39416" (error "remote error: tls: bad certificate", ServerName "")
2021-08-25 05:54:59.253553 I | embed: rejected connection from "192.168.190.78:39430" (error "remote error: tls: bad certificate", ServerName "")
RA489 commented 3 years ago

/kind support

RA489 commented 3 years ago

related to https://github.com/kubernetes/kubeadm/issues/910

neolit123 commented 3 years ago

<1.20 is out of support.

You could try refreshing the certs with 'kubeadm certs renew' before upgrade. This command is missing in older versions so download kubeadm 1.22 and use it only for that purpose. Use the 1.18 binary for the upgrade.

/support

github-actions[bot] commented 3 years ago

Hello, @prabhinmp :robot: :wave:

You seem to have troubles using Kubernetes and kubeadm. Note that our issue trackers should not be used for providing support to users. There are special channels for that purpose.

Please see: