Hello, I've run cAdvisor on EKS cluster, and found that half on metrics not present, and it depends on instances.
I'm getting following error,
W0807 10:23:50.887737 3249444 container.go:588] Failed to update stats for container "/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope": failed to get load stat for "/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope" - path "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope", error binary.Read: invalid type int32
while on other instances it works fine.
I found thats main difference between working/notworking instances is kernel version
Fails:
AMI name: amazon-eks-node-al2023-x86_64-standard-1.30-v20240703
KV: 6.1.94-99.176.amzn2023.x86_64
CR: containerd://1.7.11
Kubelet: v1.30.0-eks-036c24b
Working:
AMI Name: amazon-eks-node-1.30-v20240729
KV: 5.10.220-209.869.amzn2.x86_64
CR: containerd://1.7.11
Kubelet: v1.30.2-eks-1552ad0
Hello, I've run cAdvisor on EKS cluster, and found that half on metrics not present, and it depends on instances.
I'm getting following error,
W0807 10:23:50.887737 3249444 container.go:588] Failed to update stats for container "/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope": failed to get load stat for "/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope" - path "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73387ff3_7d68_4b26_ace5_ab12e2c91a7f.slice/cri-containerd-1948d1e50a9b535ce6babd400f919f47dd20bd36014a8b24ae0d779b9ceec6b1.scope", error binary.Read: invalid type int32
while on other instances it works fine.
I found thats main difference between working/notworking instances is kernel version
Version of cAdvisor:
v0.49.1
andv0.50.0