Closed alexhsu-petaio closed 1 year ago
Hi @alexhsu-petaio,
Could you please provide .config
for your Buildroot configuration and the same configuration file for the Linux kernel?
Hi @kolerov ,
I upload renamed config file to txt file for buildroot and linux due to upload policy ? petaio_archs38_haps_defconfig.txt petaio_haps_hs_smp_defconfig.txt
I also upload kernel patch for you. 0000-linux-haps.patch
I also tried minimal_legacy eBPF program of libbpf-bootstrap and it can work. I don't know why minimal eBPF program get "reg type unsupported for arg#0 function handle_tp#4" error during BTF check phase. I also encounter the same symptoms for some of our eBPF program.
My build machine is CentOS 8, 4.18.0-448.el8.x86_64.
Thanks Alex
@alexhsu-petaio, could you please explain how did you build minimal
step by step? Also, could you send a minimal.bpf.o
binary?
Hi @kolerov , I add libbpf-bootstrap into buildroot custom package and include this package when build linux image. Following attachment is this package and LIBBPF_BOOTSTRAP_APPS in libbpf-bootstrap.mk describe the eBPF apps want to build and install. libbpf-bootstrap-pkg.tar.gz
Following attachment include minimal.bpf.o and minimal_legacy.bpf.o for your reference. bpf-o.tar.gz
Thanks. Alex
@alexhsu-petaio Unfortunately, I cannot reproduce this error. Seems like the problem is in these warning lines:
libbpf: map 'minimal_.bss': skipped auto-creating...
libbpf: map 'minimal_.rodata': skipped auto-creating...
libbpf
cannot allocate sections for global data and BPF program cannot get necessary data. As far as I know, this problem is not related to the version of the kernel above 5.2.
Lets try from another side. I've attached a binary of minimal
which works on my side. Could you please check it on your side? Could you also send your minimal
binary.
Hi @kolerov , Thanks for your try. I have tried your mininal-to-check program and it's still fail. I attach the log of mininal-to-check program and binary of minimal_petaio program built by our site. minimal_ebpf_dbg.tar.gz
We use the same config file for kernel (6.1.15) and buildroot (2022.11.1) but get different result, it's really strange. Do you have any idea about that?
Thanks.
Alex
Hi @kolerov , I found libbpf seems use eBPF program of type BPF_PROG_TYPE_SOCKET_FILTER to check if target kernel support FEAT_GLOBAL_DATA. After enable network working support in target kernel even our plaftorm don't has network device, minimal program can create global data sections and run successfully.
Thanks for your help.
Alex
@alexhsu-petaio Glad to hear! However, some BPF programs from these examples will work incorrectly. It happens because build system of this set of examples assumes that your target is the same as your host (it includes wrong vmlinux.h
and uses host's Linux headers instead of toolchain's headers). I will explain how to avoid this problem soon.
@alexhsu-petaio Do these steps to ensure that everything is built correctly. Suppose that my Buildroot directory is buildroot
and a build directory resides in it: buildroot/build
. I'm going to place sources for libbpf-bootstrap
in build/overrides/libbpf-bootstrap
.
libbpf-bootstrap
and retrieve modules:git clone https://github.com/libbpf/libbpf-bootstrap
cd libbpf-bootstrap
git submodule update --init --recursive
libbpf-bootstrap/examples/c/Makefile
this way so we can substitute it by our own value in the future:- CLANG_BPF_SYS_INCLUDES = $(shell $(CLANG) -v -E - </dev/null 2>&1 \
+ CLANG_BPF_SYS_INCLUDES := $(shell $(CLANG) -v -E - </dev/null 2>&1 \
bpftool
is installed on your host system. Build your vmlinux
as usual. Then generate vmlinux.h
from it:
bpftool --debug btf dump file vmlinux format c > vmlinux.h
vmlinux.h
to libbpf-bootstrap/vmlinux/arc
libbpf-bootstrap.mk
for the package for Buildroot (use a corresponding path for your toolchain in CLANG_BPF_SYS_INCLUDES
):
define LIBBPF_BOOTSTRAP_BUILD_CMDS
$(TARGET_MAKE_ENV) $(TARGET_CONFIGURE_OPTS) $(MAKE) \
- -C $(LIBBPF_BOOTSTRAP_SRC_DIR) $(LIBBPF_BOOTSTRAP_APPS)
+ -C $(LIBBPF_BOOTSTRAP_SRC_DIR) $(LIBBPF_BOOTSTRAP_APPS) ARCH=$(ARCH) CLANG_BPF_SYS_INCLUDES="-I/tools/arc-linux-gnu/sysroot/usr/include"
endef
override
in buildroot/build
with this content:
LIBBPF_BOOTSTRAP_OVERRIDE_SRCDIR = build/overrides/libbpf-bootstrap
BR2_PACKAGE_OVERRIDE_FILE="build/override"
Then everything must be built correctly. If you are not sure then do these steps to check it:
make libbpf-bootstrap-dirclean
make libbpf-bootstrap V=1
If everything is ok you will see something like this (correct include path for your toolchain and correct vmlinux.h
):
clang -g -O2 -target bpf -D__TARGET_ARCH_arc -I.output -I../../libbpf/include/uapi -I../../vmlinux/arc/ -I/tools/arc-linux-gnu/sysroot/usr/include -c minimal.bpf.c -o .output/minimal.bpf.o
...
/home/ykolerov/workspace/tasks/ebpf_jit_1/buildroot/build/host/bin/arc-linux-gnu-gcc -g -Wall -I.output -I../../libbpf/include/uapi -I../../vmlinux/arc/ -c uprobe.c -o .output/uprobe.o
Hi @kolerov , Thanks a lot for your kindly shared information, it's helpful. Here has one question about item 5. For CLANG_BPF_SYS_INCLUDES="-I/tools/arc-linux-gnu/sysroot/usr/include", do you mean the path of $(STAGING_DIR)/usr/include? I can't find "tools" folder in my build environment.
Thanks. Alex
@alexhsu-petaio Yes, it is. It would be even better since you don't have to use explicit path for your toolchain.
@alexhsu-petaio So, can we consider this issue resolved? Does everything build and run fine?
Hi @kolerov , This issue has been resolved. Thanks for your help.
Alex
Hi, I run minimal eBPF program of libbpf-bootstrap and got "failed to load object 'minimal_bpf'" error. Do you have any idea about that? Here is my environment:
Following is the fail log:
./minimal
libbpf: loading object 'minimal_bpf' from buffer libbpf: elf: section(3) tp/syscalls/sys_enter_write, size 104, link 0, flags 6, type=1 libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 13 insns (104 bytes) libbpf: elf: section(4) .reltp/syscalls/sys_enter_write, size 32, link 13, flags 40, type=9 libbpf: elf: section(5) license, size 13, link 0, flags 3, type=1 libbpf: license of minimalbpf is Dual BSD/GPL libbpf: elf: section(6) .bss, size 4, link 0, flags 3, type=8 libbpf: elf: section(7) .rodata, size 28, link 0, flags 2, type=1 libbpf: elf: section(8) .BTF, size 684, link 0, flags 0, type=1 libbpf: elf: section(10) .BTF.ext, size 160, link 0, flags 0, type=1 libbpf: elf: section(13) .symtab, size 192, link 1, flags 0, type=2 libbpf: looking for externs among 8 symbols... libbpf: collected 0 externs total libbpf: map 'minimal.bss' (global data): at secidx 6, offset 0, flags 400. libbpf: map 0 is "minimal.bss" libbpf: map 'minimal_.rodata' (global data): at secidx 7, offset 0, flags 80. libbpf: map 1 is "minimal.rodata" libbpf: sec '.reltp/syscalls/sys_enter_write': collecting relocation for section(3) 'tp/syscalls/sys_enter_write' libbpf: sec '.reltp/syscalls/sys_enter_write': relo #0: insn #2 against 'my_pid' libbpf: prog 'handletp': found data map 0 (minimal.bss, sec 6, off 0) for insn 2 libbpf: sec '.reltp/syscalls/sys_enter_write': relo #1: insn #6 against '.rodata' libbpf: prog 'handletp': found data map 1 (minimal.rodata, sec 7, off 0) for insn 6 libbpf: map 'minimal.bss': skipped auto-creating... libbpf: map 'minimal.rodata': skipped auto-creating... libbpf: prog 'handletp': relo #0: poisoning insn #2 that loads map #0 'minimal.bss' libbpf: prog 'handletp': relo #1: poisoning insn #6 that loads map #1 'minimal.rodata' libbpf: prog 'handle_tp': BPF program load failed: Invalid argument libbpf: prog 'handle_tp': -- BEGIN PROG LOAD LOG -- reg type unsupported for arg#0 function handle_tp#4 0: R1=ctx(off=0,imm=0) R10=fp0 ; int pid = bpf_get_current_pid_tgid() >> 32; 0: (85) call bpf_get_current_pid_tgid#14 ; R0_w=scalar() ; int pid = bpf_get_current_pid_tgid() >> 32; 1: (77) r0 >>= 32 ; R0_w=scalar(umax=4294967295,var_off=(0x0; 0xffffffff)) ; if (pid != mypid) 2:
BPF map 'minimal .bss' is referenced but wasn't created
processed 3 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: prog 'handle_tp': failed to load: -22
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -22
Failed to load and verify BPF skeleton
Thanks. Alex