cadvisor collects lots of unneed systemd cgroups stats

[Alexbyy]

my cadvisor collects a lot of unneeded metrics,these metrics don't have a label of container_name or pod_name, and I don't want cadvisor collects these metrics. Is there any way to make the cadvisor only collect metrics which has labels of container_name and pod_name?

This is my k8s version: Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"archive", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"$Format:%H$", GitTreeState:"archive", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

[dashpole]

Since 1.15, we only collect metrics required by the summary API: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L604. So you should be able to use KubeletCgroups, RuntimeCgroups, and SystemCgroups to determine which cgroup subtrees are collected.

[Alexbyy]

Since 1.15, we only collect metrics required by the summary API: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L604. So you should be able to use KubeletCgroups, RuntimeCgroups, and SystemCgroups to determine which cgroup subtrees are collected.

Thanks for your reply.I will have a try.

[Alexbyy]

Since 1.15, we only collect metrics required by the summary API: https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/server.go#L604. So you should be able to use KubeletCgroups, RuntimeCgroups, and SystemCgroups to determine which cgroup subtrees are collected.

Hi dashpole， Does that mean that I can use the flags of "--system-cgroups"、"--kubelet-cgroups" to determine which cgroup subtrees are collected? and if I don't specify any of these flags,Could you tell me what does cadvisor collect by default?Many Thanks!

[dashpole]

The behavior looks like it is: If you don't set any of those, and cgroups-per-qos is true (which is the default IIRC), we only monitor cgroups inside /kubepods. If you set --kubelet-cgroups, we also monitor cgroups within that cgroup. If you set --system-cgroups, we also monitor cgroups within that cgroup.

[Alexbyy]

The behavior looks like it is: If you don't set any of those, and cgroups-per-qos is true (which is the default IIRC), we only monitor cgroups inside /kubepods. If you set --kubelet-cgroups, we also monitor cgroups within that cgroup. If you set --system-cgroups, we also monitor cgroups within that cgroup.

Thank you very much. I don't set any of those,and cgroups-per-qos is true,when I curl the cadvisor's port,I got this, as you can see there is not only /kubepods,but also has "id="/system.slice/docker.service""、“id="/system.slice/docker.service" etc.Is that a normal result? my k8s is v1.18.6

This is my kubelete config:

[dashpole]

Looks like you are still on 1.14 based on the initial comment, so you will probably still have all metrics.

[Alexbyy]

Looks like you are still on 1.14 based on the initial comment, so you will probably still have all metrics.

I don't think so.because the last comment is based on 1.18,not 1.14.This is my k8s version info.

[dashpole]

That looks like the version of your kubectl binary. The version of the kubelet is what matters. Check kubectl get no

[Alexbyy]

That looks like the version of your kubectl binary. The version of the kubelet is what matters. Check kubectl get no That shows the version is also v1.18.6.

[Alexbyy]

That looks like the version of your kubectl binary. The version of the kubelet is what matters. Check kubectl get no

Is that a bug or it will collect '/system.slice/docker.service' and '/system.slice/kubelet.service' by default,even though I don't set any of those?

[dashpole]

It will collect those by default, even if you don't set them. But it shouldn't collect random /system.slice/* cgroups unless you set something additional

[Alexbyy]

It will collect those by default, even if you don't set them. But it shouldn't collect random /system.slice/* cgroups unless you set something additional

ok，I see，Thank you very much!