failed to init init master0 failed, error: exit status 1. Please clean and reinstall

victory460 commented 1 year ago

Sealos Version

4.1.7

How to reproduce the bug?

No response

What is the expected behavior?

[root@XCDCP-2023-P2F1-TPOD1-PM-B1H-2 k8s_installers]# sealos run localhost/labring/kubernetes:v1.21.5 --single 2023-08-18T14:57:39 info Start to create a new cluster: master [100.70.78.2], worker [], registry 100.70.78.2 2023-08-18T14:57:39 info Executing pipeline Check in CreateProcessor. 2023-08-18T14:57:39 info checker:hostname [100.70.78.2:22] 2023-08-18T14:57:39 info checker:timeSync [100.70.78.2:22] 2023-08-18T14:57:39 info Executing pipeline PreProcess in CreateProcessor. 2023-08-18T14:57:39 info Executing pipeline RunConfig in CreateProcessor. 2023-08-18T14:57:39 info Executing pipeline MountRootfs in CreateProcessor. 2023-08-18T14:57:40 info Executing pipeline MirrorRegistry in CreateProcessor. 2023-08-18T14:57:40 info Executing pipeline Bootstrap in CreateProcessor which: no docker in (/usr/share/Modules/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/Arcconf:/root/bin) INFO [2023-08-18 14:57:47] >> check root,port,cri success chmod: cannot operate on dangling symlink '/usr/bin/policytool' Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /etc/systemd/system/containerd.service. INFO [2023-08-18 14:57:49] >> Health check containerd! INFO [2023-08-18 14:57:49] >> containerd is running INFO [2023-08-18 14:57:49] >> init containerd success chmod: cannot operate on dangling symlink '/usr/bin/policytool' Created symlink /etc/systemd/system/multi-user.target.wants/image-cri-shim.service → /etc/systemd/system/image-cri-shim.service. INFO [2023-08-18 14:57:50] >> Health check image-cri-shim! INFO [2023-08-18 14:57:50] >> image-cri-shim is running INFO [2023-08-18 14:57:50] >> init shim success

Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ... kernel.yama.ptrace_scope = 0
Applying /usr/lib/sysctl.d/50-coredump.conf ... kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h kernel.core_pipe_limit = 16 fs.suid_dumpable = 2
Applying /usr/lib/sysctl.d/50-default.conf ... kernel.sysrq = 16 kernel.core_uses_pid = 1 net.ipv4.conf.default.rp_filter = 1 sysctl: setting key "net.ipv4.conf.all.rp_filter": Invalid argument net.ipv4.conf.default.accept_source_route = 0 sysctl: setting key "net.ipv4.conf.all.accept_source_route": Invalid argument net.ipv4.conf.default.promote_secondaries = 1 sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument net.ipv4.ping_group_range = 1 0 net.core.default_qdisc = fq_codel fs.protected_hardlinks = 1 fs.protected_symlinks = 1 fs.protected_regular = 1 fs.protected_fifos = 1
Applying /usr/lib/sysctl.d/50-libkcapi-optmem_max.conf ... net.core.optmem_max = 81920
Applying /usr/lib/sysctl.d/50-pid-max.conf ... kernel.pid_max = 4194304
Applying /usr/lib/sysctl.d/60-libvirtd.conf ... fs.aio-max-nr = 1048576
Applying /etc/sysctl.d/99-sysctl.conf ... net.netfilter.nf_conntrack_max = 262144 net.nf_conntrack_max = 262144 net.ipv4.ip_forward = 1 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.tcp_keepalive_intvl = 3 net.ipv4.tcp_keepalive_time = 30 net.ipv4.tcp_keepalive_probes = 8 net.ipv4.tcp_retries2 = 5 net.ipv4.ip_forward = 1 net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 0 net.ipv4.ip_nonlocal_bind = 1 net.ipv6.conf.all.disable_ipv6 = 0 kernel.pid_max = 316448 kernel.core_pattern = /var/corefile/core-%e kernel.core_uses_pid = 1 kernel.core_pattern = /var/corefile/core-%e kernel.core_uses_pid = 1
Applying /etc/sysctl.d/k8s.conf ... net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.conf.all.rp_filter = 0
Applying /etc/sysctl.conf ... net.netfilter.nf_conntrack_max = 262144 net.nf_conntrack_max = 262144 net.ipv4.ip_forward = 1 net.ipv4.conf.default.accept_redirects = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.tcp_keepalive_intvl = 3 net.ipv4.tcp_keepalive_time = 30 net.ipv4.tcp_keepalive_probes = 8 net.ipv4.tcp_retries2 = 5 net.ipv4.ip_forward = 1 net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 0 net.ipv4.ip_nonlocal_bind = 1 net.ipv6.conf.all.disable_ipv6 = 0 kernel.pid_max = 316448 kernel.core_pattern = /var/corefile/core-%e kernel.core_uses_pid = 1 kernel.core_pattern = /var/corefile/core-%e kernel.core_uses_pid = 1 net.ipv4.ip_forward = 1 unknown system, use default to stop firewalld INFO [2023-08-18 14:57:51] >> init kube success INFO [2023-08-18 14:57:51] >> init containerd rootfs success chmod: cannot operate on dangling symlink '/usr/bin/policytool' Created symlink /etc/systemd/system/multi-user.target.wants/registry.service → /etc/systemd/system/registry.service. INFO [2023-08-18 14:57:51] >> Health check registry! INFO [2023-08-18 14:57:51] >> registry is running INFO [2023-08-18 14:57:51] >> init registry success 2023-08-18T14:57:51 info Executing pipeline Init in CreateProcessor. 2023-08-18T14:57:51 info start to copy kubeadm config to master0 2023-08-18T14:57:52 info start to generate cert and kubeConfig... 2023-08-18T14:57:52 info start to generator cert and copy to masters... 2023-08-18T14:57:52 info apiserver altNames : {map[apiserver.cluster.local:apiserver.cluster.local kubernetes:kubernetes kubernetes.default:kubernetes.default kubernetes.default.svc:kubernetes.default.svc kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local localhost:localhost xcdcp-2023-p2f1-tpod1-pm-b1h-2:xcdcp-2023-p2f1-tpod1-pm-b1h-2] map[10.103.97.2:10.103.97.2 10.96.0.1:10.96.0.1 100.70.78.2:100.70.78.2 127.0.0.1:127.0.0.1]} 2023-08-18T14:57:52 info Etcd altnames : {map[localhost:localhost xcdcp-2023-p2f1-tpod1-pm-b1h-2:xcdcp-2023-p2f1-tpod1-pm-b1h-2] map[100.70.78.2:100.70.78.2 127.0.0.1:127.0.0.1 ::1:::1]}, commonName : xcdcp-2023-p2f1-tpod1-pm-b1h-2 2023-08-18T14:57:55 info start to copy etc pki files to masters 2023-08-18T14:57:56 info start to copy etc pki files to masters 2023-08-18T14:57:56 info start to create kubeconfig... 2023-08-18T14:57:57 info start to copy kubeconfig files to masters 2023-08-18T14:57:57 info start to copy static files to masters 2023-08-18T14:57:57 info start to init master0... 2023-08-18T14:57:57 info registry auth in node 100.70.78.2:22 2023-08-18T14:57:57 info domain sealos.hub:100.70.78.2 append success 2023-08-18T14:57:57 info domain apiserver.cluster.local:100.70.78.2 append success W0818 14:57:57.509865 27175 strict.go:54] error unmarshaling configuration schema.GroupVersionKind{Group:"kubeproxy.config.k8s.io", Version:"v1alpha1", Kind:"KubeProxyConfiguration"}: error unmarshaling JSON: while decoding JSON: json: unknown field "detectLocal" W0818 14:57:57.512902 27175 strict.go:54] error unmarshaling configuration schema.GroupVersionKind{Group:"kubelet.config.k8s.io", Version:"v1beta1", Kind:"KubeletConfiguration"}: error unmarshaling JSON: while decoding JSON: json: unknown field "localStorageCapacityIsolation" W0818 14:57:57.514959 27175 utils.go:69] The recommended value for "healthzBindAddress" in "KubeletConfiguration" is: 127.0.0.1; the provided value is: 0.0.0.0 [init] Using Kubernetes version: v1.21.5 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Using existing ca certificate authority [certs] Using existing apiserver certificate and key on disk [certs] Using existing apiserver-kubelet-client certificate and key on disk [certs] Using existing front-proxy-ca certificate authority [certs] Using existing front-proxy-client certificate and key on disk [certs] Using existing etcd/ca certificate authority [certs] Using existing etcd/server certificate and key on disk [certs] Using existing etcd/peer certificate and key on disk [certs] Using existing etcd/healthcheck-client certificate and key on disk [certs] Using existing apiserver-etcd-client certificate and key on disk [certs] Using the existing "sa" key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf" [kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf" W0818 14:58:07.688007 27175 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/controller-manager.conf" exists already but has an unexpected API Server URL: expected: https://100.70.78.2:6443, got: https://apiserver.cluster.local:6443 [kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf" W0818 14:58:07.865740 27175 kubeconfig.go:246] a kubeconfig file "/etc/kubernetes/scheduler.conf" exists already but has an unexpected API Server URL: expected: https://100.70.78.2:6443, got: https://apiserver.cluster.local:6443 [kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [kubelet-check] Initial timeout of 40s passed.
```
Unfortunately, an error has occurred:
        timed out waiting for the condition

This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.

Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
        - 'crictl --runtime-endpoint /run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'crictl --runtime-endpoint /run/containerd/containerd.sock logs CONTAINERID'
```

error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster To see the stack trace of this error execute with --v=5 or higher 2023-08-18T15:02:08 error Applied to cluster error: failed to init init master0 failed, error: exit status 1. Please clean and reinstall Error: failed to init init master0 failed, error: exit status 1. Please clean and reinstall

What do you see instead?

No response

Operating environment

- Sealos version:
- Docker version:
- Kubernetes version: v1.21.5
- Operating system:
- Runtime environment:
- Cluster size:
- Additional information:

Additional information

No response

cuisongliu commented 1 year ago

升级sealos https://github.com/labring/sealos/blob/main/docs/4.0/i18n/zh-Hans/lifecycle-management/QA.md#q2报错error-fileavailable--etc-kubernetes-kubeletconf-etckuberneteskubeletconf-already-exists

sealos-ci-robot commented 1 year ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Upgrade sealos https://github.com/labring/sealos/blob/main/docs/4.0/i18n/zh-Hans/lifecycle-management/QA.md#q2 error error-fileavailable--etc-kubernetes-kubeletconf-etckuberneteskubeletconf -already-exists

SupRenekton commented 3 months ago

请问下，这个问题解决了么，我也碰到一样的问题了，用的国产麒麟sp3操作系统部署k8s出现了和你一样的问题，api-server,etcd,schedule这些静态pod拉不起来，然后kubelet报错getting node master01 not found

sealos-ci-robot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Please tell me, has this problem been solved? I also encountered the same problem. I used the domestic Kirin sp3 operating system to deploy k8s and had the same problem as you. The static pods such as api-server, etcd, and schedule could not be pulled up, and then kubelet reports error getting node master01 not found

SupRenekton commented 3 months ago

请问下，这个问题解决了么，我也碰到一样的问题了，用的国产麒麟sp3操作系统部署k8s出现了和你一样的问题，api-server,etcd,schedule这些静态pod拉不起来，然后kubelet报错getting node master01 not found

sealos-ci-robot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Please tell me, has this problem been solved? I also encountered the same problem. I used the domestic Kirin sp3 operating system to deploy k8s and had the same problem as you. The static pods such as api-server, etcd, and schedule could not be pulled up, and then kubelet reports error getting node master01 not found

labring / sealos