iovisor / bcc

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
Apache License 2.0
20.63k stars 3.89k forks source link

How to use libbpf-tools with ubuntu 18.04 4.15 kernel #3232

Open photoszzt opened 3 years ago

photoszzt commented 3 years ago

I'm new to ebpf. I saw that BPF CO-RE argument and it seems it would be the future way to use bpf. But when I try to use it on stock ubuntu 18.04. I had problems. I had other constraints so that I can't upgrade the kernel version.

I notice that the stock kernel doesn't have /sys/kernel/btf/vmlinux. I don't know how to regenerate the vmlinux.h file. But I install the debug symbol package. I think they serve similar purpose but I don't find a way to generate vmlinux.h on the README.

I try to compile the binary in libbpf-tools dir and run time. They all prompt it can't find btf info.

Based on the above problems, is that mean I should use the bcc instead? I can compile and run the python version tools.

yonghong-song commented 3 years ago

I notice that the stock kernel doesn't have /sys/kernel/btf/vmlinux. I don't know how to regenerate the vmlinux.h file. But I install the debug symbol package. I think they serve similar purpose but I don't find a way to generate vmlinux.h on the README.

You can use pahole (better latest, maybe at least >= 1.16) to generate btf with vmlinux. Not sure how easy it is to do with separate vmlinux debug symbol package. Worst case, you can get exact linux source code and config, rebuild with debug_info enabled to produce a vmlinux and then use pahole to generate btf. @anakryiko probably has more information on this.

ArivuAlamari commented 3 years ago

I'm new to ebpf. I saw that BPF CO-RE argument and it seems it would be the future way to use bpf. But when I try to use it on stock ubuntu 18.04. I had problems. I had other constraints so that I can't upgrade the kernel version.

I notice that the stock kernel doesn't have /sys/kernel/btf/vmlinux. I don't know how to regenerate the vmlinux.h file. But I install the debug symbol package. I think they serve similar purpose but I don't find a way to generate vmlinux.h on the README.

I am not sure the below steps will work for 4.15 Kernel You need the BTF enabled kernel to get the /sys/kernel/btf/vmlinux available to you. In this case you need to re-compile the Kernel with BTF enabled. But the Kernel build will look for pehole which is >= 1.16. So you need to install it before you start re-building the kernel. Following procedure would work. But please refer the repo for latest info on source code build

sudo apt-get install libelf-dev libdwarf-dev # some dependency install
git clone https://github.com/acmel/dwarves.git
cd dwarves/
git checkout tags/v1.19 -b V1.19-branch # take the latest version
mkdir build
cd build
cmake -D__LIB=lib ..
sudo make install
sudo /sbin/ldconfig -v  #may be needed when u want to update the shared library

Now you need to update the ubuntu kernel version with BTF enabled (assuming that u r building it on the target device)

sudo apt-get build-dep linux linux-image-$(uname -r) # install build dep
sudo apt-get install libncurses-dev flex bison openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev autoconf # additional tools
wget https://mirrors.edge.kernel.org/pub/linux/kernel/v5.x/linux-5.4.52.tar.xz # take the version u r interested in and change according for below commands
unxz linux-5.4.52.tar.xz
tar -xvf linux-5.4.52.tar
cd linux-5.4.52/
cp -v /boot/config-$(uname -r) .config # Starting with your existing config file will be easey
make menuconfig # search & modify the options (enable BTF Flag( DEBUG_INFO_BTF)) and save
make deb-pkg
sudo dpkg -i  ../*.deb # install the newly built kernel
# hopefully on next re-start you get the /sys/kernel/btf/vmlinux mounted

I try to compile the binary in libbpf-tools dir and run time. They all prompt it can't find btf info.

Based on the above problems, is that mean I should use the bcc instead? I can compile and run the python version tools.

anakryiko commented 3 years ago

On Sat, Jan 16, 2021 at 7:17 PM Zhiting Zhu notifications@github.com wrote:

I'm new to ebpf. I saw that BPF CO-RE argument and it seems it would be the future way to use bpf. But when I try to use it on stock ubuntu 18.04. I had problems. I had other constraints so that I can't upgrade the kernel version.

I notice that the stock kernel doesn't have /sys/kernel/btf/vmlinux. I don't know how to regenerate the vmlinux.h file. But I install the debug symbol package. I think they serve similar purpose but I don't find a way to generate vmlinux.h on the README.

You can use BPF CO-RE with very old kernels that never supported BTF, but there are a lot of nuances to get this working. Please see [0] and [1] for recent discussion on how to generate BTF on your own and provide it to libbpf. So if you absolutely must do it on 4.15 you can with a bunch of extra work setting everything up. But of course it's much simpler on a bit more recent kernels. Ubuntu 20.10 has everything pre-setup for BPF CO-RE, so that would be the recommended way, of course.

Also, if you need to work with just your one specific kernel, you don't need BPF CO-RE, you can just use system headers to compile everything.

[0] https://lore.kernel.org/bpf/CADmGQ+1euj7Uv9e8UyZMMXDiYAKqXe9=GSTBFNbbg1E0R-ejyg@mail.gmail.com/ [1] https://lore.kernel.org/bpf/B8801F77-37E8-4EF8-8994-D366D48169A3@araalinetworks.com/

I try to compile the binary in libbpf-tools dir and run time. They all prompt it can't find btf info.

Based on the above problems, is that mean I should use the bcc instead? I can compile and run the python version tools.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/iovisor/bcc/issues/3232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4BK7N5TTK74C637D4ZA3S2JJFTANCNFSM4WFYHVFQ .

photoszzt commented 3 years ago

Thanks for the pointer. I will look at how to use libbpf without using the CO-RE.