search

weaveworks / scope

Monitoring, visualisation & management for Docker & Kubernetes
https://www.weave.works/oss/scope/
Apache License 2.0
5.86k stars 714 forks source link

Failure in eBPF tracker: /sys/kernel/debug/tracing/kprobe_events: file exists #3786

Closed bboreham closed 4 years ago

bboreham commented 4 years ago

What happened?

<probe> WARN: 2020/05/07 16:30:46.704903 Error setting up the eBPF tracker, falling back to proc scanning: cannot write "p:ptcp_v4_connect tcp_v4_connect\n" to kprobe_events: write /sys/kernel/debug/tracing/kprobe_events: file exists

The symptom goes away on reboot, until you run a second time. It seems as though Scope is not clearing down the kprobes when it exits?

How to reproduce it?

scope launch scope stop scope launch

Versions:

This is on Ubuntu 20.04

$ uname -a
Linux vagrant 5.4.0-29-generic #33-Ubuntu SMP Wed Apr 29 14:32:27 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
bboreham commented 4 years ago

I fixed this by waiting for the tracer to exit. Suggests that we still have a problem if the probe crashes mid-run.

CermakM commented 3 years ago

I am still hitting this issue. Can you elaborate a bit more how you fixed it, please?

bboreham commented 3 years ago

Did you get it after exactly the steps outlined in the description? That is what is fixed. As I said, "we still have a problem if the probe crashes mid-run".