search

kubernetes / kubeadm

Aggregator for issues filed against kubeadm
Apache License 2.0
3.77k stars 717 forks source link

Error when joining the node to the cluster, when using both 1GB and 2MB huge-page sizes are enabled on the node #1785

Closed Alxsmith11 closed 5 years ago

Alxsmith11 commented 5 years ago

Is this a request for help?

If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.

If no, delete this section and continue on.

What keywords did you search in kubeadm issues before filing this one?

If you have found any duplicates, you should instead reply there and close this page.

If you have not found any duplicates, delete this section and continue on.

Is this a BUG REPORT or FEATURE REQUEST?

Choose one: BUG REPORT or FEATURE REQUEST

Versions

kubeadm version (use kubeadm version): kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:11:18Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

Environment:

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.3 LTS Release: 16.04 Codename: xenial

What happened?

Node failed to join the cluster

hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) none on /dev/hugepages1G type hugetlbfs (rw,relatime,pagesize=1G) none on /dev/hugepages2M type hugetlbfs (rw,relatime,pagesize=2M)

kubeadm join 192.168.211.226:6443 --token ea1g9s.gsli4aevgc304u04 \

--discovery-token-ca-cert-hash sha256:65c6dda82eb3a54f4285e31cde4352869b66b070d29011b6faca583acca12e39

[preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Activating the kubelet service [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... [kubelet-check] Initial timeout of 40s passed. error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition

systemctl status kubelet

● kubelet.service - kubelet: The Kubernetes Node Agent Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/kubelet.service.d └─10-kubeadm.conf Active: active (running) since Tue 2019-09-10 08:53:11 PDT; 3min 39s ago Docs: https://kubernetes.io/docs/home/ Main PID: 3909 (kubelet) Tasks: 49 Memory: 51.4M CPU: 4.984s CGroup: /system.slice/kubelet.service └─3909 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs -

Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.388436 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.488606 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.588746 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.688999 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.789222 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.889406 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:50 ubuntu-211224 kubelet[3909]: E0910 08:56:50.989603 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:51 ubuntu-211224 kubelet[3909]: E0910 08:56:51.089823 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:51 ubuntu-211224 kubelet[3909]: E0910 08:56:51.190042 3909 kubelet.go:2248] node "ubuntu-211224" not found Sep 10 08:56:51 ubuntu-211224 kubelet[3909]: E0910 08:56:51.290255 3909 kubelet.go:2248] node "ubuntu-211224" not found lines 1-23/23 (END)

What you expected to happen?

Node should have join

How to reproduce it (as minimally and precisely as possible)?

  1. Enable 16Gi in the 1GB huge-page table, if Numa is enabled 8 on each node
  2. Enable 16Gi in the 2MB huge-page table, if Numa is enabled 8 on each node
  3. Kubeadm join xxxxx

    Anything else we need to know?

neolit123 commented 5 years ago

hi, try logging the same issue in the kubernetes/kubernetes repository. huge pages is k8s specific and not maintained or tested by the kubeadm team.

thanks.

Alxsmith11 commented 5 years ago

The problem is when doing kubeadm join?