openyurtio / openyurt

OpenYurt - Extending your native Kubernetes to edge(project under CNCF)
https://openyurt.io
Apache License 2.0
1.71k stars 401 forks source link

coredns无法解析,kube-proxy endpointslists请求数无法释放 #2167

Open zhangchi6414 opened 1 week ago

zhangchi6414 commented 1 week ago

已经安装文档给的提示加入到了nodepool中

openyurt-helm版本1.5.0 raven kube-system 1 2024-10-11 22:42:14.090677672 +0800 CST deployed raven-agent-0.4.2 0.4.2 yurt-hub kube-system 1 2024-10-11 22:40:41.510098368 +0800 CST deployed yurthub-1.5.0 1.5.0 yurt-manager kube-system 1 2024-10-11 22:38:58.065591264 +0800 CST deployed yurt-manager-1.5.0 1.5.0

kubernetes version root@sz-lt-i08-manager03:/home/megaium# kubectl version Client Version: v1.28.2 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.28.2

查看endpointslists root@server9:/home/megaium# cat /etc/kubernetes/cache/kube-proxy/endpointslices.v1.discovery.k8s.io/kube-system/kube-dns-vrr8q |jq . { "kind": "EndpointSlice", "apiVersion": "discovery.k8s.io/v1", "metadata": { "name": "kube-dns-vrr8q", "generateName": "kube-dns-", "namespace": "kube-system", "uid": "dbb73d26-f741-4e62-9ee7-0bf8a73a858a", "resourceVersion": "3209748", "generation": 404, "creationTimestamp": "2024-09-30T11:37:12Z", "labels": { "endpointslice.kubernetes.io/managed-by": "endpointslice-controller.k8s.io", "k8s-app": "kube-dns", "kubernetes.io/cluster-service": "true", "kubernetes.io/name": "KubeDNS", "kubernetes.io/service-name": "kube-dns" }, "annotations": { "endpoints.kubernetes.io/last-change-trigger-time": "2024-10-12T06:34:57Z" }, "ownerReferences": [ { "apiVersion": "v1", "kind": "Service", "name": "kube-dns", "uid": "02ca64b1-517a-4c3d-ac2a-6def0bb1cef0", "controller": true, "blockOwnerDeletion": true } ], "managedFields": [ { "manager": "kube-controller-manager", "operation": "Update", "apiVersion": "discovery.k8s.io/v1", "time": "2024-10-12T06:34:57Z", "fieldsType": "FieldsV1", "fieldsV1": { "f:addressType": {}, "f:endpoints": {}, "f:metadata": { "f:annotations": { ".": {}, "f:endpoints.kubernetes.io/last-change-trigger-time": {} }, "f:generateName": {}, "f:labels": { ".": {}, "f:endpointslice.kubernetes.io/managed-by": {}, "f:k8s-app": {}, "f:kubernetes.io/cluster-service": {}, "f:kubernetes.io/name": {}, "f:kubernetes.io/service-name": {} }, "f:ownerReferences": { ".": {}, "k:{\"uid\":\"02ca64b1-517a-4c3d-ac2a-6def0bb1cef0\"}": {} } }, "f:ports": {} } } ] }, "addressType": "IPv4", "endpoints": [ { "addresses": [ "192.168.1.38" ], "conditions": { "ready": true, "serving": true, "terminating": false }, "targetRef": { "kind": "Pod", "namespace": "kube-system", "name": "coredns-n9xvn", "uid": "c692f328-41ab-4646-8ac8-8c92544f9ee1" }, "nodeName": "cx3001237923792379" } ], "ports": [ { "name": "metrics", "protocol": "TCP", "port": 9153 }, { "name": "dns", "protocol": "UDP", "port": 53 }, { "name": "dns-tcp", "protocol": "TCP", "port": 53 } ] }

node_yurthub_in_flight_requests_collector请求 ` root@server9:/home/megaium# curl http://127.0.0.1:10267/metrics |grep node_yurthub_proxy_traffic_collector % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 55027 0 55027 # HELP node_yurthub_proxy_traffic_collector collector of proxy response traffic by hub agent(unit: byte) 0 # TYPE node_yurthub_proxy_traffic_collector counter 0 36.node_yurthub_proxy_traffic_collector{client="coredns",resource="endpointslices",subresources="",verb="list"} 1.189629e+06 0M node_yurthub_proxy_traffic_collector{client="coredns",resource="endpointslices",subresources="",verb="watch"} 22954 node_yurthub_proxy_traffic_collector{client="coredns",resource="namespaces",subresources="",verb="list"} 201045 node_yurthub_proxy_traffic_collector{client="coredns",resource="namespaces",subresources="",verb="watch"} 1444 0node_yurthub_proxy_traffic_collector{client="coredns",resource="services",subresources="",verb="list"} 1.221222e+06 -node_yurthub_proxy_traffic_collector{client="coredns",resource="services",subresources="",verb="watch"} 4478 -:node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="endpointslices",subresources="",verb="list"} 34933 --:node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="endpointslices",subresources="",verb="watch"} 108596 -- node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="events",subresources="",verb="create"} 991 -node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="nodes",subresources="",verb="get"} 12859 -:-node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="nodes",subresources="",verb="list"} 12903 -:-node_yurthub_proxy_traffic_collector{client="kube-proxy",resource="nodes",subresources="",verb="watch"} 8.555354e+06

` 多个边缘节点,目前只有该节点会出现这个问题,现在环境还存在可以提供

luc99hen commented 5 days ago

Please post the log of yurthub.