Closed shuiouyfm closed 2 years ago
chaudron
commented
2 years ago Looks like your kernel does not support multi-program loading. Do you get the same error if you do not supply the -p option? Can you dump the full output of -vv if this also does not work, i.e. :
xdpdump -i veth9ebc34d -vv
shuiouyfm
commented
2 years ago hi, thanks your reply. the output as follows:
[feyuan@10.224.170.142 ~/xdp-tools/xdp-dump]$sudo ./xdpdump -i veth9ebc34d -vv
Current rlimit 67108864 already >= minimum 1048576
libxdp: DATASEC 'xdp_metadata' not found.
libxdp: DATASEC '.xdp_run_config' not found.
libxdp: Found legacy program with id 1941 and 0 component progs
Found func xdp_sock_prog matching xdp_sock_prog
Looking for './xdpdump_bpf.o'
Loading bpf file 'xdpdump_bpf.o' from './xdpdump_bpf.o'
libbpf: loading ./xdpdump_bpf.o
libbpf: elf: section(3) fentry/func, size 304, link 0, flags 6, type=1
libbpf: sec 'fentry/func': found program 'trace_on_entry' at insn offset 0 (0 bytes), code size 38 insns (304 bytes)
libbpf: elf: section(4) .relfentry/func, size 32, link 26, flags 0, type=9
libbpf: elf: section(5) fexit/func, size 320, link 0, flags 6, type=1
libbpf: sec 'fexit/func': found program 'trace_on_exit' at insn offset 0 (0 bytes), code size 40 insns (320 bytes)
libbpf: elf: section(6) .relfexit/func, size 32, link 26, flags 0, type=9
libbpf: elf: section(7) license, size 4, link 0, flags 3, type=1
libbpf: license of ./xdpdump_bpf.o is GPL
libbpf: elf: section(8) .maps, size 32, link 0, flags 3, type=1
libbpf: elf: section(9) .data, size 12, link 0, flags 3, type=1
libbpf: elf: section(18) .BTF, size 1855, link 0, flags 0, type=1
libbpf: elf: section(20) .BTF.ext, size 1132, link 0, flags 0, type=1
libbpf: elf: section(26) .symtab, size 1848, link 1, flags 0, type=2
libbpf: looking for externs among 77 symbols...
libbpf: collected 0 externs total
libbpf: map 'xdpdump_perf_map': at sec_idx 8, offset 0.
libbpf: map 'xdpdump_perf_map': found type = 4.
libbpf: map 'xdpdump_perf_map': found key [3], sz = 4.
libbpf: map 'xdpdump_perf_map': found value [10], sz = 4.
libbpf: map 'xdpdump_perf_map': found maxentries = 256.
libbpf: map 'xdpdump.data' (global data): at secidx 9, offset 0, flags 400.
libbpf: map 1 is "xdpdump.data"
libbpf: sec '.relfentry/func': collecting relocation for section(3) 'fentry/func'
libbpf: sec '.relfentry/func': relo #0: insn #7 against 'trace_cfg'
libbpf: prog 'trace_onentry': found data map 1 (xdpdump.data, sec 9, off 0) for insn 7
libbpf: sec '.relfentry/func': relo #1: insn #32 against 'xdpdump_perf_map'
libbpf: prog 'trace_on_entry': found map 0 (xdpdump_perf_map, sec 8, off 0) for insn #32
libbpf: sec '.relfexit/func': collecting relocation for section(5) 'fexit/func'
libbpf: sec '.relfexit/func': relo #0: insn #8 against 'trace_cfg'
libbpf: prog 'trace_onexit': found data map 1 (xdpdump.data, sec 9, off 0) for insn 8
libbpf: sec '.relfexit/func': relo #1: insn #34 against 'xdpdump_perf_map'
libbpf: prog 'trace_on_exit': found map 0 (xdpdump_perf_map, sec 8, off 0) for insn #34
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: map 'xdpdump_perfmap': created successfully, fd=5
libbpf: map 'xdpdump.data': created successfully, fd=6
libbpf: sec 'fentry/func': found 6 CO-RE relocations
libbpf: CO-RE relocating [17] struct xdp_buff: found target candidate [27985] struct xdp_buff in [vmlinux]
libbpf: prog 'trace_on_entry': relo #0: kind
chaudron
commented
2 years ago Not sure what happens here. Quickly browsing the kernel code I could be the program has too many instructions, but I think we can rule that out. Or there is something that goes wrong with copying the data from userspace in libbpf.
@tohojo do you know anything about libbpf compatibility problems?
tohojo
commented
2 years ago @tohojo do you know anything about libbpf compatibility problems?
Nope :(
chaudron
commented
2 years ago @shuiouyfm did you build with the embedded libbpf or with the system one?
If you're lib/libbpf directory has no content, you build with the system included one. To try and build with the embedded one, you can do FORCE_SUBDIR_LIBBPF=1 ./configure and rebuild.
If you do not have the system libbpf library installed, can you try installing those, and do another ./configure, you can remove the ./lib/libbpf submodule to make sure its not re-used.
shuiouyfm
commented
2 years ago @chaudron, as your memtioned , my kernel version is 5.4 and I found some info as :
In other words, kernels before 5.6 can only attach a single XDP program to each interface, kernels 5.6+ can attach multiple programs if they are all attached at the same time, and kernels 5.10 have full support for XDP multiprog on x86_64. from https://www.mankier.com/3/libxdp#Kernel_and_BPF_program_feature_compatibility
I guess this is the root cause.
And I build with the embedded lib/libbpf.
thank you.
chaudron
commented
2 years ago I missed the kernel part :( So for XDP dump the kernel needs to at least have the below three commits. And from the git repo data, you need at least 5.10 also.
$ git tag --contains d831ee84bfc9 | grep -v rc | sort -n
v5.10
v5.11
v5.12
v5.13
v5.14
v5.15
v5.16
v5.7
v5.8
v5.9
$ git tag --contains 5b92a28aae4d | grep -v rc | sort -n
v5.10
v5.11
v5.12
v5.13
v5.14
v5.15
v5.16
v5.5
v5.6
v5.7
v5.8
v5.9
$ git tag --contains 43bc2874e779 | grep -v rc | sort -n
v5.10
v5.11
v5.12
v5.13
v5.14
v5.15
v5.16So I guess you can close this issue, due to an unsupported kernel version.
tohojo
commented
2 years ago Eelco Chaudron @.***> writes:
So I guess you can close this issue, due to an unsupported kernel version.
But shouldn't xdpdump print a nice error message about this now? So why isn't it?
chaudron
commented
2 years ago Well, the error might not be because of the missing commits, looks more like a compatibility issue with libbpf, but not 100% sure. With the missing patches, I think I've seen different error messages in the past.
tohojo
commented
2 years ago So should we do something like this?
diff --git a/xdp-dump/xdpdump.c b/xdp-dump/xdpdump.c
index cf201e830a2f..48c4c0cf8771 100644
--- a/xdp-dump/xdpdump.c
+++ b/xdp-dump/xdpdump.c
@@ -1419,7 +1419,7 @@ rlimit_loop:
trace_link_fentry = bpf_program__attach_trace(trace_prog_fentry);
err = libbpf_get_error(trace_link_fentry);
if (err) {
- if (err == -ENOTSUPP)
+ if (err == -ENOTSUPP || err == -E2BIG)
#if defined(__x86_64__) || defined(__i686__)
pr_warn("ERROR: The kernel does not support "
"fentry function attach because it is "@tohojo I looked a bit at the code again, and I do not see a real problem that could indicate a real problem that reports E2BIG for the load program. So to solve this we need to add this check to the "err = bpf_object__load(trace_obj);" part of the code, not the attach part.
tohojo
commented
2 years ago @shuiouyfm could you please test the patch in #179 and see if you get a nicer error message with that? :)
shuiouyfm
commented
2 years ago hi the error message is printed in my case [feyuan@10.224.170.142 ~/xdp-tools/xdp-dump]$sudo ./xdpdump -i veth9ebc34d -p xdp_sock_prog -v -x Current rlimit 67108864 already >= minimum 1048576 Found func xdp_sock_prog matching xdp_sock_prog Looking for './xdpdump_bpf.o' Loading bpf file 'xdpdump_bpf.o' from './xdpdump_bpf.o' ERROR: The kernel does not support fentry function load because it is too old!
tohojo
commented
2 years ago shuiouyfm @.***> writes:
hi the error message is printed in my case @.*** ~/xdp-tools/xdp-dump]$sudo ./xdpdump -i veth9ebc34d -p xdp_sock_prog -v -x Current rlimit 67108864 already >= minimum 1048576 Found func xdp_sock_prog matching xdp_sock_prog Looking for './xdpdump_bpf.o' Loading bpf file 'xdpdump_bpf.o' from './xdpdump_bpf.o' ERROR: The kernel does not support fentry function load because it is too old!
Awesome! Thanks for testing!
hi,
Interface Prio Program name Mode ID Tag Chain actions
lo <No XDP program loaded!> ens1f0 <No XDP program loaded!> enp0s31f6 <No XDP program loaded!> veth9ebc34d xdp_sock_prog native 1941 84dbbb323171de50 veth929ee17 <No XDP program loaded!>
Do I make a mistake of it ? how can I do next step?