This repository is archived. Please file in-tree vSphere Cloud Provider issues at https://github.com/kubernetes/kubernetes/issues . CSI Driver for vSphere is available at https://github.com/kubernetes/cloud-provider-vsphere
Apache License 2.0
46
stars
31
forks
source link
Nodes configured with VCP are failing to update their status on v1.11.0 #494
After upgrading to v1.11.0 with kubeadm following these instructions, nodes configured with VCP are failing to update their status, causing them to become NotReady, unless --node-ip is explicitely specified.
What you expected to happen:
Nodes becoming Ready.
How to reproduce it (as minimally and precisely as possible):
Upgrade from v.10.0 to v1.11.0 via kubeadm
Reconfigure config files with VCP options
Nodes cannot become ready
Remove --cloud-provider=vsphere from kubelet arguments and restart kubelet
Nodes can become ready
Add --cloud-provider=vsphere --node-ip to kubelet arguments and restart
Nodes can become ready
Anything else we need to know?:
kubelet logs are showing these messages:
Jun 29 10:14:10 node4 kubelet[1901]: a256:d5b9f663a00615de7b9c1084586a4b83f40343cee86699204e0989b49a9ab459\",\"k8s.gcr.io/kube-apiserver-amd64:v1.10.1\"],\"sizeBytes\":225028923}],\"volumesInUse\":[\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-023a94ec-4e1e-11e8-ba0a-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-0eac5821-7b00-11e8-9c39-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-0eaeea82-7b00-11e8-9c39-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-0eb21340-7b00-11e8-9c39-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-19ed5306-64dc-11e8-91a5-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-365eea96-4a53-11e8-b23c-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-60c3c27b-4e08-11e8-ba0a-0050569e6be4.vmdk\",\"kubernetes.io/vsphere-volume/[K8SDS] kubevols/kubernetes-dynamic-pvc-d21a6695-69ba-11e8-91a5-0050569e6be4.vmdk\"]}}" for node "node4": The order in patch list:
Jun 29 10:14:10 node4 kubelet[1901]: [map[address:172.25.53.24 type:ExternalIP] map[type:ExternalIP address:172.28.72.194] map[type:InternalIP address:172.25.53.24] map[address:172.28.72.194 type:InternalIP]]
Jun 29 10:14:10 node4 kubelet[1901]: doesn't match $setElementOrder list:
Jun 29 10:14:10 node4 kubelet[1901]: [map[type:ExternalIP] map[type:InternalIP] map[type:ExternalIP] map[type:InternalIP] map[type:Hostname]]
Jun 29 10:14:10 node4 kubelet[1901]: E0629 10:14:10.247923 1901 kubelet_node_status.go:379] Unable to update node status: update node status exceeds retry count
ESXi 6.0 and later (VM version 11)
VMware Tools: version:2147483647
OS (e.g. from /etc/os-release):
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"
Kernel (e.g. uname -a):
Linux node4 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Dec 28 14:23:39 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
/kind bug
What happened:
After upgrading to v1.11.0 with kubeadm following these instructions, nodes configured with VCP are failing to update their status, causing them to become NotReady, unless --node-ip is explicitely specified.
What you expected to happen:
Nodes becoming Ready.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
kubelet logs are showing these messages:
Environment:
kubectl version):Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
ESXi 6.0 and later (VM version 11) VMware Tools: version:2147483647
NAME="Red Hat Enterprise Linux Server" VERSION="7.4 (Maipo)" ID="rhel" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="7.4" PRETTY_NAME="Red Hat Enterprise Linux" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server" HOME_URL="https://www.redhat.com/" BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" REDHAT_BUGZILLA_PRODUCT_VERSION=7.4 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="7.4"
uname -a):Linux node4 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Dec 28 14:23:39 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
Install tools: kubeadm
Others: