OpenWrt Support

[PolynomialDivision]

I try to cross-compile your tool for OpenWrt. Since u use reallocarray and similar functions you enable #define _GNU_SOURCE. I would assume that everyhing is correct, but it still fails to compile.

params.c: In function 'handle_multistring':
params.c:58:8: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration]
  ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),
        ^~~~~~~~~~~~
        realloc
params.c:58:6: error: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion]
  ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),

[PolynomialDivision]

The config is set. So the error makes no sense: https://github.com/openwrt/openwrt/blob/master/target/linux/generic/config-5.4#L652

[tohojo]

Polynomdivision notifications@github.com writes:

The config is set. So the error makes no sense: https://github.com/openwrt/openwrt/blob/master/target/linux/generic/config-5.4#L652

Notice the lines immediately after it: libbpf: Error in bpf_object__probe_loading():Operation not permitted(1). Couldn't load trivial BPF program. Make sure your kernel supports BPF (CONFIG_BPF_SYSCALL=y) and/or that RLIMIT_MEMLOCK is set to big enough value. libbpf: failed to load object './xdpfilt_alw_all.o' Permission denied when loading eBPF object; raising rlimit and retrying Doubling current rlimit of 4194304

So it just does that a couple of times while xdp-tools is raising the rlimit...

[PolynomialDivision]

So it just does that a couple of times while xdp-tools is raising the rlimit...

Any idea?

BTW: blogic now added an own loader for OpenWrt: https://github.com/blogic/ubpf

[tohojo]

Polynomdivision notifications@github.com writes:

So it just does that a couple of times while xdp-tools is raising the rlimit...

Any idea?

As far as I can tell the only error you have left is the one that tells your the kernel is too old... We should probably add a compatibility mode for that as long as you're only loading a single program...

[PolynomialDivision]

As far as I can tell the only error you have left is the one that tells your the kernel is too old...

Fortunately, I saw kernel 5.10 is added to OpenWrt :)

• https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=3a50e86172f9503a7306db8ebcdb1ea3c26980a4
• https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=6d6e6b2bef54fbf324272d6faf9aec89d662e416

I have to look for a mediatek Router.

[tohojo]

Polynomdivision notifications@github.com writes:

As far as I can tell the only error you have left is the one that tells your the kernel is too old...

Fortunately, I saw kernel 5.10 is added to OpenWrt :)

• https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=3a50e86172f9503a7306db8ebcdb1ea3c26980a4
• https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=6d6e6b2bef54fbf324272d6faf9aec89d662e416

I have to look for a mediatek Router.

In the meantime, I just pushed an update with the compatibility fallback, so you should be able to load programs even on older kernels (you won't get multi-prog capability, obviously, and it'll warn about missing features).

[PolynomialDivision]

Polynomdivision notifications@github.com writes:

As far as I can tell the only error you have left is the one that tells your the kernel is too old... Fortunately, I saw kernel 5.10 is added to OpenWrt :) - https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=3a50e86172f9503a7306db8ebcdb1ea3c26980a4 - https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=6d6e6b2bef54fbf324272d6faf9aec89d662e416 I have to look for a mediatek Router. In the meantime, I just pushed an update with the compatibility fallback, so you should be able to load programs even on older kernels (you won't get multi-prog capability, obviously, and it'll warn about missing features).

Nice! I will directly test.

[PolynomialDivision]

Still have the error on x86_64?

root@OpenWrt:~# xdp-loader load -vv eth0 /root/srv6_remover.o 
Setting rlimit to minimum 1048576
Loading 4277729 files on interface 'eth0'.
libbpf: loading ELF
libbpf: failed to open ELF: No such file or directory
libxdp: Couldn't load the eBPF program (libbpf said 'no such file').
Maybe the program was compiled with a too old version of LLVM (need v9.0+)?
Couldn't open file 'ELF': No such file or directory
Segmentation fault

[PolynomialDivision]

Maybe it is the cmd-parsing again?

[PolynomialDivision]

Okay, num_progs is just again a huge number, that is derived from opt->filenames.num_strings;

[tohojo]

Polynomdivision notifications@github.com writes:

Maybe it is the cmd-parsing again?

Yeah, it looks like it's trying to open a file called "ELF", which is nonsensical. I just pushed a fix Eelco noticed for the config parsing, but I am not sure if this is your issue. I do wonder if you need to set some more variables in your package Makefile; for instance, the configure script needs variables set rather than parameters. I think OpenWrt lets you set those by appending to CONFIGURE_VARS in the Makefile. Could you post the full output of a clean package build somewhere?

[PolynomialDivision]

Could you post the full output of a clean package build somewhere?

Yes of course. But could u please specify what u exactly mean with "clean package build"?

[tohojo]

Polynomdivision notifications@github.com writes:

Could you post the full output of a clean package build somewhere?

Yes of course. But could u please specify what u exactly mean with "clean package build"?

Something like:

make package/xdp-tools/clean && make package/xdp-tools/compile V=s

[tohojo]

Polynomdivision notifications@github.com writes:

Could you post the full output of a clean package build somewhere?

Yes of course. But could u please specify what u exactly mean with "clean package build"?

Also, please add V=1 to MAKE_FLAGS in your package Makefile so the xdp-tools compilation goes into verbose mode...

[PolynomialDivision]

Polynomdivision notifications@github.com writes:

Could you post the full output of a clean package build somewhere? Yes of course. But could u please specify what u exactly mean with "clean package build"? Also, please add V=1 to MAKE_FLAGS in your package Makefile so the xdp-tools compilation goes into verbose mode...

Like that? https://gist.github.com/PolynomialDivision/9218248cdb0c0cd72db3f38d4447beb8

[tohojo]

Polynomdivision notifications@github.com writes:

Polynomdivision notifications@github.com writes:

Could you post the full output of a clean package build somewhere? Yes of course. But could u please specify what u exactly mean with "clean package build"? Also, please add V=1 to MAKE_FLAGS in your package Makefile so the xdp-tools compilation goes into verbose mode...

Like that? https://gist.github.com/PolynomialDivision/9218248cdb0c0cd72db3f38d4447beb8

Yes, thank you!

Well, for one thing it is not installing the xdp-dispatcher.o anywhere as far as I can tell. What's the output of 'opkg files xdp-tools' after you install?

Also, for the num_strings issue, could you please try applying this patch:

diff --git a/xdp-loader/xdp-loader.c b/xdp-loader/xdp-loader.c
index fe52c1b..cf0c7ae 100644
--- a/xdp-loader/xdp-loader.c
+++ b/xdp-loader/xdp-loader.c
@@ -27,6 +27,8 @@ static const struct loadopt {
        char *section_name;
        enum xdp_attach_mode mode;
 } defaults_load = {
+       .iface = { 0 },
+       .filenames = { 0 },
        .mode = XDP_MODE_NATIVE
 };

[PolynomialDivision]

Also, for the num_strings issue, could you please try applying this patch:

Does not help. :/

Well, for one thing it is not installing the xdp-dispatcher.o anywhere

Currently I copy the xdp_dispatcher manually to the binary. I was not sure where to put it.

[PolynomialDivision]

I added some patches: https://github.com/PolynomialDivision/openwrt-xdp-tools/tree/master/xdp-tools/patches

Maybe I destroyed something?

[tohojo]

Polynomdivision notifications@github.com writes:

I added some patches: https://github.com/PolynomialDivision/openwrt-xdp-tools/tree/master/xdp-tools/patches

Maybe I destroyed something?

I don't think your patches broke anything, but most of them should be unnecessary if you call configure correctly (which the openwrt build system doesn't quite do by default). Have a look at what the RPM spec file does for build and install:

https://github.com/xdp-project/xdp-tools/blob/master/packaging/rpm/xdp-tools.spec#L64

You need to convince the OpenWrt build system to do something similar; part of it can be done by setting CONFIGURE_VARS, I think, but not sure what is needed for the install.

Your build log output also indicates that you're currently building the BPF programs using the system LLVM installation. This is probably not ideal since it means that the package won't build on systems without a clang installation. Not sure if OpenWrt includes LLVM in its toolchain at all, but figuring out how this is handled in other packages (and setting $LLC and $CLANG appropriately on configure) may be worthwhile.

As for the opt parser bug, I'm still totally stumped on that. I'll dig some more, but probably won't have time to it until after the holidays...

[PolynomialDivision]

I will fix my makefile.

Not sure if OpenWrt includes LLVM in its toolchain at all, but figuring out how this is handled in other packages (and setting $LLC and $CLANG appropriately on configure) may be worthwhile.

I'm not aware that OpenWrt ships with clang. That is why I introduced this endianess flag, to compile for the correct architecture. I will have a look again.

As for the opt parser bug, I'm still totally stumped on that. I'll dig some more, but probably won't have time to it until after the holidays..

Just ping me. Maybe I have the time to look into this, too.

Thanks again for your great support!

[PolynomialDivision]

XDP-Tools is ideally for compiling kernel modules. :) I added now some ip-stats-xdp into my openwrt xdp feed: https://github.com/PolynomialDivision/openwrt-xdp-tools/tree/master/xdp-tools

I have some issues, with adding new stuff instead of overriding things. ^^ https://github.com/PolynomialDivision/openwrt-xdp-tools/blob/master/xdp-tools/Makefile#L46-L47

Is it possible, to add a userspace program that consists of multiple files? I already tried different flags, but not completely sure.

[PolynomialDivision]

Good news!

More and more OpenWrt Devs are working on XDP! :D https://github.com/openwrt/packages/pull/12949#issuecomment-765817250

[PolynomialDivision]

No-one else is working on OpenWrt support, or? Maybe I have to to look at this again, soon.

[tohojo]

Polynomdivision notifications@github.com writes:

No-one else is working on OpenWrt support, or? Maybe I have to to look at this again, soon.

Not actively, AFAIK. I may get around to it eventually, but feel free to take another stab at it! :)

[PolynomialDivision]

Not actively, AFAIK. I may get around to it eventually, but feel free to take another stab at it! :)

I will go for it as soon as I have time. I think it would be very useful to have xdp-tools ported to openwrt. Then I can finally remove my hacky versions.

[PolynomialDivision]

BTW https://github.com/openwrt/openwrt/pull/3855

[PolynomialDivision]

FYI: @dangowrt is working on gcc10 as bpf compiler to replace llvm/clang for openwrt: https://github.com/openwrt/openwrt/pull/3855#issuecomment-864931751

[omarerc]

Polynomdivision notifications@github.com writes: I added some patches: https://github.com/PolynomialDivision/openwrt-xdp-tools/tree/master/xdp-tools/patches Maybe I destroyed something? I don't think your patches broke anything, but most of them should be unnecessary if you call configure correctly (which the openwrt build system doesn't quite do by default). Have a look at what the RPM spec file does for build and install: https://github.com/xdp-project/xdp-tools/blob/master/packaging/rpm/xdp-tools.spec#L64 You need to convince the OpenWrt build system to do something similar; part of it can be done by setting CONFIGURE_VARS, I think, but not sure what is needed for the install. Your build log output also indicates that you're currently building the BPF programs using the system LLVM installation. This is probably not ideal since it means that the package won't build on systems without a clang installation. Not sure if OpenWrt includes LLVM in its toolchain at all, but figuring out how this is handled in other packages (and setting $LLC and $CLANG appropriately on configure) may be worthwhile. As for the opt parser bug, I'm still totally stumped on that. I'll dig some more, but probably won't have time to it until after the holidays...

Hello Gentlemen, I just notice that in the new OpenWRT version 22.03.0-rc4 is included LLVM-BPF-13.0.0 (llvm-bpf-13.0.0.Linux-x86_64.tar.xz.

I am trying to use OpenWRT to use XDP to take statistics of my Traffic Network, do you guys think that this Tool can be used to solve it?

[dangowrt]

Now that we got include/bpf.mk we can build xdp-tools for starters. Review and testing anyone? https://github.com/openwrt/openwrt/pull/10139

[omarerc]

Thanks a lot. I really want to get into it.

Do you have any documentation about how to compile this packet in OpenWRT?

https://github.com/openwrt/openwrt/tree/c73471cddc5409111638d0e61e41314f5d1b8e5d/package/network/utils/xdp-tools