iovisor / bcc

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
Apache License 2.0
20.36k stars 3.86k forks source link

some libbpf-tools tooling / libbpf seems broken on thinlto-compiled kernel #5007

Open nekopsykose opened 4 months ago

nekopsykose commented 4 months ago

reproduced on current c93a19aaf3f7eae9d6c9070309cc785c18575767

$ doas ./btrfsslower 
libbpf: prog 'file_read_entry': failed to create kprobe 'btrfs_file_read_iter+0x0' perf event: No such file or directory
failed to attach kprobe: -2
failed to attach BPF programs: -2
$ doas cat /proc/kallsyms | grep btrfs_file_read_iter
ffffffffc142eb60 t btrfs_file_read_iter.llvm.3433979326929198294    [btrfs]

as far as i can tell this is because the symbols contain the llvm suffix which is what happens when CONFIG_LTO_CLANG_THIN=y. libbpf seems to have a similar fix for something like this https://github.com/libbpf/libbpf/commit/d2f83fb976fb4685fdec174ea5be57ae38bb960c

..but this fix is not involved in the codepath that at least *slower takes (all *slower tools fail like this), so there must be something else too.

i don't know what else to debug (most tools work fine, the manual bcc btrfsslower.py works if you replace the symbol name with the full name from kallsyms, ..) so i assume there is just something that is missing this suffix handling.

the tail of an strace looks like

open("/sys/bus/event_source/devices/kprobe/type", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 15
fcntl(15, F_SETFD, FD_CLOEXEC)          = 0
fcntl(15, F_SETFD, FD_CLOEXEC)          = 0
read(15, "8\n", 1024)                   = 2
read(15, "", 1024)                      = 0
close(15)                               = 0
open("/sys/bus/event_source/devices/kprobe/type", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 15
fcntl(15, F_SETFD, FD_CLOEXEC)          = 0
fcntl(15, F_SETFD, FD_CLOEXEC)          = 0
read(15, "8\n", 1024)                   = 2
read(15, "", 1024)                      = 0
close(15)                               = 0
perf_event_open({type=0x8 /* PERF_TYPE_??? */, size=0x88 /* PERF_ATTR_SIZE_??? */, config=0, sample_period=0, sample_type=0, read_format=0, precise_ip=0 /* arbitrary skid */, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = -1 ENOENT (No such file or directory)
writev(2, [{iov_base="libbpf: prog 'file_read_entry': failed to create kprobe 'btrfs_file_read_iter+0x", iov_len=80}, {iov_base="0", iov_len=1}], 2libbpf: prog 'file_read_entry': failed to create kprobe 'btrfs_file_read_iter+0x0) = 81

if you want a full one (-s 1000000), here you go (26M) https://img.ayaya.dev/g12f7mUpOvSP

in case some kernel feature is missing, this is the full Kconfig: https://img.ayaya.dev/rowG70D7QFK8