iovisor / bcc

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
Apache License 2.0
20.59k stars 3.88k forks source link

How to detach uprobes with invalid `/proc/fs` binary path after process terminates? #4843

Open benkilimnik opened 11 months ago

benkilimnik commented 11 months ago

Say we have a uprobe attached to a binary accessed via /proc/<program-pid>/root/normal/path/to/binary. When the program terminates, the pid gets cleaned up so that the /proc/fs filepath is no longer valid. To clean up the dangling FD, we need to call detach_uprobe, but this fails because check_binary_symbol can't read the file.

Is there a way to make the detach API work with proc/fs binary paths after the pid is deleted?

benkilimnik commented 11 months ago

Another way of phrasing this question: how do we detach uprobes for short-lived processes/containers? What is the expected way to manage the uprobe lifecycle on containerized (ephemeral) applications?

etep commented 10 months ago

Could a BCC maintainer please comment on this?

It would be neat to have BCC track the open probes so that the binary can be removed from the file system and the probe can still be detached. For example, BCC could return a probe descriptor which internally could map to the open FDs. When the user calls detach probe, internally BCC uses the probe descriptor to find the aforesaid FDs and then closes them.

yonghong-song commented 10 months ago

For uprobe, current bcc has a mapping 'ev_name -> fd'. When detaching happens, bcc will follow 'ev_name -> fd' mapping and find the 'fd' and do detaching. I need to double check whether 'fd' (created at 'attach'/perf_event_open point) is still valid or not for short-lived process. Do you have an example to show the issue?

etep commented 10 months ago

We will craft an example. Thank you!