coroot / coroot-node-agent

A Prometheus exporter based on eBPF that gathers comprehensive container metrics
https://coroot.com/docs/metrics/node-agent
Apache License 2.0
314 stars 56 forks source link

k0s v1.26.1+k0s.0 failed to inspect container #10

Closed irvanmohamad closed 1 year ago

irvanmohamad commented 1 year ago

I'm having trouble getting service maps working. I installed coroot into a 1 master node + 2 worker nodes cluster, all applications show external endpoints, and no CPU/Memory data is picked up.

I used helm which installed the following coroot versions:

$ helm install --namespace coroot --create-namespace coroot coroot/coroot

image: ghcr.io/coroot/coroot-node-agent:1.6.4
image: ghcr.io/coroot/coroot:0.13.1
$ k0s version
v1.26.1+k0s.0

$ uname -a
Linux kmaster 5.15.0-60-generic #66-Ubuntu SMP Fri Jan 20 14:29:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ sudo sysctl -a | grep bpf
kernel.bpf_stats_enabled = 1
kernel.unprivileged_bpf_disabled = 0
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 0
net.core.bpf_jit_kallsyms = 1
net.core.bpf_jit_limit = 264241152

node agent logs are showing failed to get container metadata for pid and failed to inspect container errors

I0226 11:40:37.256313   13256 registry.go:191] TCP connection from unknown container {connection-open none 14216 10.244.0.218:46266 10.103.101.131:80 25 5635465385867 <nil>}
W0226 11:40:37.256387   13256 registry.go:244] failed to get container metadata for pid 14216 -> /kubepods/burstable/podd785437d-e85d-40f0-b13f-52a66f1dda5d/cc5347fb09a49ab8a1017960f8c70e4e765dedb561cb0e2eb7325196fc4efcf4: failed to interact with dockerd (%!s(<nil>)) or with containerd (%!s(<nil>))

It seems that I could not push my code and create pull request, I got this error message: Permission to coroot/coroot-node-agent.git denied to irvanmohamad

I would like to add support for k0s distribution in containerd.go file, so that it would be like this:

sockets := []string{"/var/snap/microk8s/common/run/containerd.sock", "/run/k3s/containerd/containerd.sock", "/run/containerd/containerd.sock", "/run/k0s/containerd.sock"}
apetruhin commented 1 year ago

@irvanmohamad, thank you for the report. Please update coroot-node-agent to version 1.6.5 (the helm chart has also been updated).

irvanmohamad commented 1 year ago

Thank you for the update. I have tested my k0s cluster with the new helm chart and it worked like a charm 🥳