[BUG] yurtadm join failed:dial tcp 127.0.0.1:10261: connect: connection refused

kubenetes version : v1.28.0 openyurt versin : yurtadm-v1.4.4

What happened: 在边缘机器，执行openYurt join失败：xx.xx.xx.xx is my ip execute openyurt join xx.xx.xx.xx:6443 --token=4etrp2.p101y8dgjm98b9b0 --node-type=edge --discovery-token-unsafe-skip-ca-verification --v=5 --cri-socket=/run/containerd/containerd.sock but its failed: execution phase kubelet-start: error uploading crisocket: Get "http://127.0.0.1:10261/api/v1/nodes/gm-system-product-name?timeout=10s": dial tcp 127.0.0.1:10261: connect: connection refused

logs: I0704 14:29:08.852401 39105 token.go:105] [discovery] Created cluster-info discovery client, requesting info from "xx.xx.xx.xx:6443" I0704 14:29:08.852421 39105 token.go:230] [discovery] kube client is nil, use bootstrap config(&{ {false map[]} map[kubernetes:0xc0007a1700] map[] map[token-bootstrap-client@kubernetes:0xc00079e5a0] token-bootstrap-client@kubernetes map[]}) to create client I0704 14:29:08.876802 39105 token.go:130] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "xx.xx.xx.xx:6443" I0704 14:29:08.898081 39105 kubernetes.go:369] kubernetes version: v1.28.0 I0704 14:29:08.898140 39105 join.go:396] node join data info: join.joinData{cfgPath:"", joinNodeData:(joindata.NodeRegistration)(0xc00079e550), apiServerEndpoint:"xx.xx.xx.xx6443", token:"4etrp2.p101y8dgjm98b9b0", tlsBootstrapCfg:(api.Config)(0xc00057a5a0), clientSet:(*kubernetes.Clientset)(0xc0006002c0), ignorePreflightErrors:sets.String{"DirAvailable--etc-kubernetes-manifests":sets.Empty{}, "FileAvailable--etc-kubernetes-kubelet.conf":sets.Empty{}}, organizations:"", pauseImage:"registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2", yurthubImage:"registry.cn-hangzhou.aliyuncs.com/openyurt/yurthub:latest", yurthubTemplate:"", yurthubManifest:"", kubernetesVersion:"v1.28.0", caCertHashes:[]string{}, nodeLabels:map[string]string{}, kubernetesResourceServer:"dl.k8s.io", yurthubServer:"127.0.0.1", reuseCNIBin:false, namespace:"kube-system", staticPodTemplateList:[]string(nil), staticPodManifestList:[]string(nil)} I0704 14:29:08.904489 39105 system.go:43] Setting ipv4 forward I0704 14:29:08.904567 39105 system.go:52] Setting bridge settings for kubernetes. I0704 14:29:08.904709 39105 system.go:74] Disabling SELinux. I0704 14:29:08.904721 39105 kubernetes.go:117] Check and install kubelet v1.28.0 I0704 14:29:08.942215 39105 kubernetes.go:126] kubelet --version: v1.28.0 I0704 14:29:08.942243 39105 kubernetes.go:138] Kubelet v1.28.0 already exist, skip install. I0704 14:29:08.942246 39105 kubernetes.go:208] Check and install kubeadm v1.28.0 I0704 14:29:08.961953 39105 kubernetes.go:215] "kubeadm" version="{\n \"clientVersion\": {\n \"major\": \"1\",\n \"minor\": \"28\",\n \"gitVersion\": \"v1.28.0\",\n \"gitCommit\": \"855e7c48de7388eb330da0f8d9d2394ee818fb8d\",\n \"gitTreeState\": \"clean\",\n \"buildDate\": \"2023-08-15T10:20:15Z\",\n \"goVersion\": \"go1.20.7\",\n \"compiler\": \"gc\",\n \"platform\": \"linux/amd64\"\n }\n}\n" I0704 14:29:08.962009 39105 kubernetes.go:232] Kubeadm v1.28.0 already exist, skip install. I0704 14:29:08.962021 39105 kubernetes.go:183] Skip download cni, use already exist file: /tmp/cni-plugins-linux-amd64-v0.8.0.tgz I0704 14:29:09.237897 39105 kubernetes.go:258] Setting kubelet service. I0704 14:29:09.241044 39105 token.go:105] [discovery] Created cluster-info discovery client, requesting info from "xx.xx.xx.xx:6443" I0704 14:29:09.241050 39105 token.go:230] [discovery] kube client is nil, use bootstrap config(&{ {false map[]} map[kubernetes:0xc000022080] map[] map[token-bootstrap-client@kubernetes:0xc00079e9b0] token-bootstrap-client@kubernetes map[]}) to create client I0704 14:29:09.247318 39105 token.go:130] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "xx.xx.xx.xx:6443" I0704 14:29:09.247611 39105 yurthub.go:44] [join-node] Adding edge hub static yaml I0704 14:29:09.247652 39105 yurthub.go:88] yurthub template: /etc/kubernetes/manifests/yurthub.yaml apiVersion: v1 kind: Pod metadata: annotations: openyurt.io/static-pod-hash: 6c64fbdf49 creationTimestamp: null labels: k8s-app: yurt-hub name: yurt-hub namespace: kube-system spec: containers:

command:
- yurthub
- --v=2
- --bind-address=127.0.0.1
- --server-addr=https://xx.xx.xx.xx:6443
- --node-name=$(NODE_NAME)
- --bootstrap-file=/var/lib/yurthub/bootstrap-hub.conf
- --working-mode=edge
- --namespace=kube-system env:
- name: NODE_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: spec.nodeName image: openyurt/yurthub:v1.4.0 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: host: 127.0.0.1 path: /v1/healthz port: 10267 scheme: HTTP initialDelaySeconds: 300 periodSeconds: 5 successThreshold: 1 timeoutSeconds: 1 name: yurt-hub resources: limits: memory: 300Mi requests: cpu: 150m memory: 150Mi securityContext: capabilities: add:
  - NET_ADMIN
  - NET_RAW terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts:
- mountPath: /var/lib/yurthub name: hub-dir
- mountPath: /etc/kubernetes name: kubernetes dnsPolicy: ClusterFirst hostNetwork: true priority: 2000001000 priorityClassName: system-node-critical restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 volumes:
hostPath: path: /var/lib/yurthub type: DirectoryOrCreate name: hub-dir
hostPath: path: /etc/kubernetes type: Directory name: kubernetes status: {} I0704 14:29:09.247681 39105 yurthub.go:93] [join-node] Add hub agent static yaml is ok I0704 14:29:09.247686 39105 token.go:105] [discovery] Created cluster-info discovery client, requesting info from "xx.xx.xx.xx:6443" I0704 14:29:09.247689 39105 token.go:230] [discovery] kube client is nil, use bootstrap config(&{ {false map[]} map[kubernetes:0xc0005ffe00] map[] map[token-bootstrap-client@kubernetes:0xc0000b6500] token-bootstrap-client@kubernetes map[]}) to create client I0704 14:29:09.253500 39105 token.go:130] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "xx.xx.xx.xx:6443" W0704 14:29:09.300763 39154 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration! [preflight] Running pre-flight checks [WARNING FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... [kubelet-check] Initial timeout of 40s passed. error execution phase kubelet-start: error uploading crisocket: Get "http://127.0.0.1:10261/api/v1/nodes/gm-system-product-name?timeout=10s": dial tcp 127.0.0.1:10261: connect: connection refused To see the stack trace of this error execute with --v=5 or higher Error: exit status 1

kubelet logs:

7月 04 14:32:22 gm-System-Product-Name kubelet[39330]: E0704 14:32:22.512666 39330 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"http://127.0.0.1:10261/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/gm-system-product-name?timeout=10s\": dial tcp 127.0.0.1:10261: connect: connection refused" interval="7s" 7月 04 14:32:22 gm-System-Product-Name kubelet[39330]: I0704 14:32:22.668020 39330 kubelet_node_status.go:70] "Attempting to register node" node="gm-system-product-name" 7月 04 14:32:22 gm-System-Product-Name kubelet[39330]: E0704 14:32:22.668664 39330 kubelet_node_status.go:92] "Unable to register node with API server" err="Post \"http://127.0.0.1:10261/api/v1/nodes\": dial tcp 127.0.0.1:10261: connect: connection refused" node="gm-system-product-name" 7月 04 14:32:24 gm-System-Product-Name kubelet[39330]: W0704 14:32:24.117514 39330 reflector.go:535] vendor/k8s.io/client-go/informers/factory.go:150: failed to list v1.CSIDriver: Get "http://127.0.0.1:10261/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 127.0.0.1:10261: connect: connection refused 7月 04 14:32:24 gm-System-Product-Name kubelet[39330]: E0704 14:32:24.117633 39330 reflector.go:147] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch v1.CSIDriver: failed to list v1.CSIDriver: Get "http://127.0.0.1:10261/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 127.0.0.1:10261: connect: connection refused 7月 04 14:32:24 gm-System-Product-Name kubelet[39330]: E0704 14:32:24.943753 39330 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"gm-system-product-name\" not found" 7月 04 14:32:25 gm-System-Product-Name kubelet[39330]: E0704 14:32:25.234966 39330 event.go:289] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"gm-system-product-name.17deeeb1be6e9311", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:, DeletionGracePeriodSeconds:(int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"gm-system-product-name", UID:"gm-system-product-name", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"InvalidDiskCapacity", Message:"invalid capacity 0 on image filesystem", Source:v1.EventSource{Component:"kubelet", Host:"gm-system-product-name"}, FirstTimestamp:time.Date(2024, time.July, 4, 14, 29, 14, 873647889, time.Local), LastTimestamp:time.Date(2024, time.July, 4, 14, 29, 14, 873647889, time.Local), Count:1, Type:"Warning", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(v1.EventSeries)(nil), Action:"", Related:(v1.ObjectReference)(nil), ReportingController:"kubelet", ReportingInstance:"gm-system-product-name"}': 'Post "http://127.0.0.1:10261/api/v1/namespaces/default/events": dial tcp 127.0.0.1:10261: connect: connection refused'(may retry after sleeping) 7月 04 14:32:29 gm-System-Product-Name kubelet[39330]: E0704 14:32:29.513976 39330 controller.go:146] "Failed to ensure lease exists, will retry" err="Get \"http://127.0.0.1:10261/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/gm-system-product-name?timeout=10s\": dial tcp 127.0.0.1:10261: connect: connection refused" interval="7s" 7月 04 14:32:29 gm-System-Product-Name kubelet[39330]: I0704 14:32:29.670373 39330 kubelet_node_status.go:70] "Attempting to register node" node="gm-system-product-name" 7月 04 14:32:29 gm-System-Product-Name kubelet[39330]: E0704 14:32:29.670958 39330 kubelet_node_status.go:92] "Unable to register node with API server" err="Post \"http://127.0.0.1:10261/api/v1/nodes\": dial tcp 127.0.0.1:10261: connect: connection refused" node="gm-system-product-name" 7月 04 14:32:34 gm-System-Product-Name kubelet[39330]: E0704 14:32:34.944939 39330 eviction_manager.go:258] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"gm-system-product-name\" not found"

Environment:

OpenYurt version:yurtadm-v1.4.4
Kubernetes version (use kubectl version):v.1.28.0
OS (e.g: cat /etc/os-release):Ubuntu 24.02

/kind bug

openyurtio / openyurt

[BUG] yurtadm join failed:dial tcp 127.0.0.1:10261: connect: connection refused #2092