aya-rs / aya

Aya is an eBPF library for the Rust programming language, built with a focus on developer experience and operability.
https://aya-rs.dev/book/
Apache License 2.0
3.04k stars 267 forks source link

Implement machinery to resolve userspace symbols #705

Open addisoncrump opened 1 year ago

addisoncrump commented 1 year ago

Currently, the userspace symbol resolution is a bit janky because it expects the symbols to be at a fixed position, which isn't true in the case of dynamic libraries. Aya should handle this by implementing the appropriate machinery for inspecting proc maps of pids vis-a-vis bcc: https://github.com/iovisor/bcc/blob/master/examples/tracing/stack_buildid_example.py

If there is absolutely zero interest in this besides me, I'll look into implementing it.

addisoncrump commented 1 year ago

Want to give a go at this, but was told:

ISTR a discussion about this ages ago... and I think we said "this should probably live in another crate not in Aya core"

With this in mind, I propose the following:

  1. Introduce the SymbolResolver trait with a function, resolve, which accepts a mutable stacktrace
  2. change StackTrace::resolve to accept a SymbolResolver instead of a BTreeMap
  3. implement SymbolResolver for the BTreeMap to maintain backwards compatibility

Then, in a separate crate, implement ProcSymbolizer and the related ProcSymbolResolver. ProcSymbolizer will implement a function ProcSymbolizer::for_pid<'a>(&'a mut self, pid: u32) -> ProcSymbolResolver<'a> which will attempt to load the symbols from the files specified in /proc/<pid>/maps and commit them to a shared HashMap<PathBuf, BTreeMap<u64, String>> before instantiating ProcSymbolResolver<'a> which can be used to resolve the symbols for a particular pid. Similarly, provide an asynchronous AsyncProcSymbolizer and AsyncProcSymbolResolver which provides the same function (except &self and no lifetime for AsyncProcSymbolResolver and internally stores the mappings in Arc<RwLock<HashMap<PathBuf, Arc<BTreeMap<u64, String>>>>> for which the subset of key-value pairs are cloned for the AsyncProcSymbolResolver to use.

ajwerner commented 1 year ago

Another reason the logic is janky is that if you do specify the name of a library and a pid (https://github.com/aya-rs/aya/blob/b1bf61ca61285bc2390b3b3d10ee6a91eabfef34/aya/src/programs/uprobe.rs#L89-L92), then it does go look at the proc maps, but it does it in a pretty ad-hoc way (https://github.com/aya-rs/aya/blob/b1bf61ca61285bc2390b3b3d10ee6a91eabfef34/aya/src/programs/uprobe.rs#L232-L265). Then, if it finds a match for the library, it requires you to have used ldconfig to do the loading, and it will fail if /etc/ld.so.cache doesn't exist: https://github.com/aya-rs/aya/blob/b1bf61ca61285bc2390b3b3d10ee6a91eabfef34/aya/src/programs/uprobe.rs#L102-L106

This is also janky if a binary links a library with the same name as itself dynamically.

All of this is to say, I am very supportive of this being pulled out.

yshui commented 6 months ago

Also note NixOS does not use /etc/ld.so.cache at all.