内存使用量显示不正常的问题

tiansin commented 5 months ago

k8s版本v1.23.15

shaxiaozz commented 5 months ago

@tiansin kubepi版本多少？

tiansin commented 5 months ago

@shaxiaozz kubepi 版本: v1.7.0

shaxiaozz commented 5 months ago

复现不出来

tiansin commented 5 months ago

@shaxiaozz 我是拿最新版本的kubekey装的,不会是因为这个吧.

shaxiaozz commented 5 months ago

@tiansin 镜像v1.7.0是正常的

tiansin commented 5 months ago

1718770322270 @shaxiaozz 但我这就是不正常,奇怪了... 这问题有没有什么排查方法呢? 系统是 AlmaLinux 9.4

shaxiaozz commented 5 months ago

@tiansin 你4个节点渲染都不行吗？

tiansin commented 5 months ago

@shaxiaozz 节点用起来没有问题,都能正常使用的.就是显示的数据不正常

tiansin commented 5 months ago

节点列表里边的显示也是错的

tiansin commented 5 months ago

@shaxiaozz

/kubepi/api/v1/proxy/prod-k8s/k8s/api/v1/nodes?search=true&pageNum=1&pageSize=10 这个接口返回的部分json数据:

正常显示的:

                "allocatable": {
                    "cpu": "4",
                    "ephemeral-storage": "483173420666",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "16162352Ki",
                    "pods": "110"
                },
                "capacity": {
                    "cpu": "4",
                    "ephemeral-storage": "524276716Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "16264752Ki",
                    "pods": "110"
                },

不不正常的:

                "allocatable": {
                    "cpu": "3600m",
                    "ephemeral-storage": "208394220Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "7265135610",
                    "pods": "110"
                },
                "capacity": {
                    "cpu": "4",
                    "ephemeral-storage": "208394220Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "8007220Ki",
                    "pods": "110"
                },

shaxiaozz commented 5 months ago

@tiansin 我这里查看是正常的

tiansin commented 5 months ago

@shaxiaozz 我去分析了下代码,发现原因是我这边返回的usage.memory和status.allocatable.memory属性返回的是byte类型数值. https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/business/cluster/nodes/index.vue#L172

导致下边的两个函数处理有问题. https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/filters/index.js#L81 https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/utils/unitConvert.js#L15

shaxiaozz commented 5 months ago

@tiansin 你可以查看下你的metrics-server版本是多少吗

shaxiaozz commented 5 months ago

@tiansin 你可以按照以下方法使用curl请求k8s api-server接口，测试验证吗？

Step 1

cd ~/.kube/
cat config | grep certificate-authority-data | awk '{print $2}' | base64 -d > ca.crt
cat config | grep client-certificate-data | awk '{print $2}' | base64 -d > client.crt
cat config | grep client-key-data | awk '{print $2}' | base64 -d > client.key
APISERVER=$(cat config | grep server | awk '{print $2}')

Step 2

curl --cert ./client.crt --cacert ./ca.crt --key ./client.key $APISERVER/apis/metrics.k8s.io/v1beta1/nodes

我的metrics-server版本是v0.5.0

tiansin commented 4 months ago

@tiansin 你可以查看下你的metrics-server版本是多少吗

tiansin commented 4 months ago

@tiansin 你可以按照以下方法使用curl请求k8s api-server接口，测试验证吗？

Step 1

cd ~/.kube/
cat config | grep certificate-authority-data | awk '{print $2}' | base64 -d > ca.crt
cat config | grep client-certificate-data | awk '{print $2}' | base64 -d > client.crt
cat config | grep client-key-data | awk '{print $2}' | base64 -d > client.key
APISERVER=$(cat config | grep server | awk '{print $2}')

Step 2

curl --cert ./client.crt --cacert ./ca.crt --key ./client.key $APISERVER/apis/metrics.k8s.io/v1beta1/nodes

我的metrics-server版本是v0.5.0

{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {

  },
  "items": [
    {
      "metadata": {
        "name": "k8s-master",
        "creationTimestamp": "2024-06-21T04:35:54Z",
        "labels": {
          "beta.kubernetes.io/arch": "amd64",
          "beta.kubernetes.io/os": "linux",
          "kubernetes.io/arch": "amd64",
          "kubernetes.io/hostname": "k8s-master",
          "kubernetes.io/os": "linux",
          "node-role.kubernetes.io/control-plane": "",
          "node-role.kubernetes.io/master": "",
          "node.kubernetes.io/exclude-from-external-load-balancers": ""
        }
      },
      "timestamp": "2024-06-21T04:35:42Z",
      "window": "20s",
      "usage": {
        "cpu": "149009900n",
        "memory": "3203244Ki"
      }
    },
    {
      "metadata": {
        "name": "k8s-node1",
        "creationTimestamp": "2024-06-21T04:35:54Z",
        "labels": {
          "beta.kubernetes.io/arch": "amd64",
          "beta.kubernetes.io/os": "linux",
          "kubernetes.io/arch": "amd64",
          "kubernetes.io/hostname": "k8s-node1",
          "kubernetes.io/os": "linux",
          "node-role.kubernetes.io/worker": ""
        }
      },
      "timestamp": "2024-06-21T04:35:41Z",
      "window": "20s",
      "usage": {
        "cpu": "271930850n",
        "memory": "16294076Ki"
      }
    },
    {
      "metadata": {
        "name": "k8s-node2",
        "creationTimestamp": "2024-06-21T04:35:54Z",
        "labels": {
          "beta.kubernetes.io/arch": "amd64",
          "beta.kubernetes.io/os": "linux",
          "kubernetes.io/arch": "amd64",
          "kubernetes.io/hostname": "k8s-node2",
          "kubernetes.io/os": "linux",
          "node-role.kubernetes.io/worker": ""
        }
      },
      "timestamp": "2024-06-21T04:35:38Z",
      "window": "10s",
      "usage": {
        "cpu": "166265u",
        "memory": "16454776Ki"
      }
    },
    {
      "metadata": {
        "name": "k8s-node3",
        "creationTimestamp": "2024-06-21T04:35:54Z",
        "labels": {
          "beta.kubernetes.io/arch": "amd64",
          "beta.kubernetes.io/os": "linux",
          "kubernetes.io/arch": "amd64",
          "kubernetes.io/hostname": "k8s-node3",
          "kubernetes.io/os": "linux",
          "node-role.kubernetes.io/worker": ""
        }
      },
      "timestamp": "2024-06-21T04:35:44Z",
      "window": "20s",
      "usage": {
        "cpu": "190991150n",
        "memory": "17670484Ki"
      }
    }
  ]
}

shaxiaozz commented 4 months ago

@tiansin 看了。kubepi不会响应数据，node memory的数据都是从apiserver获取的。对应两个接口 https://192.168.56.101:6443/api/v1/nodes?search=true&pageNum=1&pageSize=100 https://192.168.56.101:6443/apis/metrics.k8s.io/v1beta1/nodes

直接原封不动的将apiserver获取到数据返回的

tiansin commented 4 months ago

@shaxiaozz 这个我也分析过了,返回的数据是对的,只是我这里返回的是纯数字的Byte数值,而不是带Ki这种. 然后下边的两个格式化函数,一个未处理未返回,另一个直接Number(value)返回

这张图就是直接Number(value)返回,所以是个很大的值.

这张图就是未处理,导致Gi前面的值都是空的.

@shaxiaozz 我去分析了下代码,发现原因是我这边返回的usage.memory和status.allocatable.memory属性返回的是byte类型数值.

https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/business/cluster/nodes/index.vue#L172

导致下边的两个函数处理有问题.

https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/filters/index.js#L81

https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/utils/unitConvert.js#L15

shaxiaozz commented 4 months ago

@tiansin 你的节点是不是有超分配的情况。k8s的接口是会带上单位返回的，不带单位返回本来就是一个异常

shaxiaozz commented 4 months ago

@tiansin 你可以请求/api/v1/nodes?search=true&pageNum=1&pageSize=100看看allocatable.memory是否不存在单位

tiansin commented 4 months ago

@shaxiaozz

/kubepi/api/v1/proxy/prod-k8s/k8s/api/v1/nodes?search=true&pageNum=1&pageSize=10 这个接口返回的部分json数据:

正常显示的:

                "allocatable": {
                    "cpu": "4",
                    "ephemeral-storage": "483173420666",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "16162352Ki",
                    "pods": "110"
                },
                "capacity": {
                    "cpu": "4",
                    "ephemeral-storage": "524276716Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "16264752Ki",
                    "pods": "110"
                },

不不正常的:

                "allocatable": {
                    "cpu": "3600m",
                    "ephemeral-storage": "208394220Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "7265135610",
                    "pods": "110"
                },
                "capacity": {
                    "cpu": "4",
                    "ephemeral-storage": "208394220Ki",
                    "hugepages-1Gi": "0",
                    "hugepages-2Mi": "0",
                    "memory": "8007220Ki",
                    "pods": "110"
                },

正常显示的那个是我的一个老服务器,不正常显示的是我最近新搭的服务器.

@shaxiaozz 并没有超分配,用kubekey弄出来就是这样,这个我之前不是发出来了了嘛

wanghe-fit2cloud commented 3 months ago

v1.8.0 版本已修复该问题。

tiansin commented 3 months ago

v1.8.0 版本已修复该问题。

感谢,另外的那个显示问题可以也修复一下吗?

tiansin commented 3 months ago

@shaxiaozz @wanghe-fit2cloud 这里边 giMemoryFormat 和 miMemoryFormat 函数也有处理不了byte值

https://github.com/1Panel-dev/KubePi/blob/0209ad00eed505f4f1b1c5bca2942708665984b8/web/dashboard/src/filters/index.js#L81

1Panel-dev / KubePi

内存使用量显示不正常的问题 #205