Closed namecloudz closed 10 months ago
Hey! What Linux distro are you using when building the program and can you give the output of the following?
# Kernel version
uname -r
# Release file
cat /etc/*-release
I've ran into this error before, but I don't remember what I did to fix it and I'm pretty sure it's related to LibXDP (missing a dependency).
If you want something quickly, you can try checking out the commit before moving to LibXDP which is detailed here. There haven't been many changes to the XDP code itself, so you should be running the same program with the exception of using LibXDP to load the BPF/XDP program.
root@filter:~# uname -r
5.4.0-139-generic
root@filter:~# cat /etc/*-release
DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS" NAME="Ubuntu" VERSION="20.04.6 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.6 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal
I spun up a vanilla Ubuntu 20.04 VM with the same kernel version as you and ran into the same issue. I couldn't even compile bpftool
from source and the issue for me ended up being the kernel being too outdated. I was able to get it working by installing kernel image linux-image-5.15.0-86-generic
which ships with Ubuntu 20.04 and its headers via the following command.
sudo apt install linux-image-5.15.0-86-generic linux-headers-5.15.0-86-generic linux-tools-5.15.0-86-generic
I tried installing linux-tools-generic
and linux-tools-5.15.0-86-generic
, but I had the same error. However, I was able to compile bpftool
successfully from source which fixed the issue.
# Clone BPFTool and its submodules.
git clone --recurse-submodules https://github.com/libbpf/bpftool.git
# Change directory to source.
cd bpftool/src
# Make and install.
make -j 2
sudo make install
I also installed version 14 of clang
and had to manually create a symbolic link for llc
. I'm not entirely sure version >= 14 is needed. I'm going to reinstall the VM soon and see if the kernel change plus version 12 of clang
which ships with Ubuntu 20.04 is fine enough.
# Download and use LVM 14 install script.
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 14
# Replace older version of Clang with 14.
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-14 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-14 100
# Remove old LLC link and use LLVM/LLC-14 instead.
sudo rm -f /usr/bin/llc
sudo ln -s /usr/lib/llvm-14/bin/llc /usr/bin/llc
I hope this helps and I'll let you know once I reinstall the VM!
After a full reinstall of the VM, I was able to get away with installing kernel image 5.15.0-86-generic
and Clang 12. Clang 10 was too outdated, but it's possible Clang 11 may work (I didn't test Clang 11).
I didn't need to build bpftool
from source as well, so I installed the kernel 5.15.0-86-generic
tools as suggested in @pettai's linked issue :smile:
Here are the commands I performed from the start of the vanilla VM:
# Update and upgrade packages
sudo apt update
sudo apt upgrade
# Install kernel 5.15.0-86-generic and its tools for bpftool
sudo apt install -y linux-image-5.15.0-86-generic linux-headers-5.15.0-86-generic linux-tools-5.15.0-86-generic
# Reboot to load new kernel
sudo reboot
# Confirm kernel version
uname -r
# Install dependencies for XDP Firewall
sudo apt install -y libconfig-dev llvm clang libelf-dev build-essential
sudo apt install -y libpcap-dev m4 gcc-multilib
# Install Clang 12
sudo apt install -y clang-12
# Update primary clang binaries to use version 12
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-12 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 100
# Confirm Clang version
clang -v
# Replace LLVM/LLC 10 with version 14 (otherwise an error occurs when building).
# Note - I'm not sure what to use with `update-alternatives` since `llc` and `llvm` doesn't work with it. So instead, I manually remove the old symbolic link and create a new one pointed towards LLVM 12.
sudo rm -f /usr/bin/llc
sudo ln -s /usr/lib/llvm-12/bin/llc /usr/bin/llc
# Confirm LLVM/LLC version
llc --version
# Clone XDP Firewall and its sub-modules
git clone --recursive https://github.com/gamemann/XDP-Firewall.git
# Change directory
cd XDP-Firewall
# Make and install LibXDP
make libxdp
# Make XDP Firewall
make
# Install Firewall
sudo make install
# Edit /etc/xdpfw/xdpfw.conf if needed (e.g. using different interface name)
# Run Firewall
sudo xdpfw
When building, I do receive a warning, but I believe it's harmless.
clang -lconfig -lelf -lz -I modules/xdp-tools/lib/libbpf/src -I /usr/include -I /usr/local/include -o build/xdpfw modules/xdp-tools/lib/libbpf/src/staticobjs/bpf_prog_linfo.o modules/xdp-tools/lib/libbpf/src/staticobjs/bpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf_dump.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf.o modules/xdp-tools/lib/libbpf/src/staticobjs/gen_loader.o modules/xdp-tools/lib/libbpf/src/staticobjs/hashmap.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_errno.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_probes.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/linker.o modules/xdp-tools/lib/libbpf/src/staticobjs/netlink.o modules/xdp-tools/lib/libbpf/src/staticobjs/nlattr.o modules/xdp-tools/lib/libbpf/src/staticobjs/relo_core.o modules/xdp-tools/lib/libbpf/src/staticobjs/ringbuf.o modules/xdp-tools/lib/libbpf/src/staticobjs/str_error.o modules/xdp-tools/lib/libbpf/src/staticobjs/strset.o modules/xdp-tools/lib/libbpf/src/staticobjs/usdt.o modules/xdp-tools/lib/libxdp/sharedobjs/xsk.o modules/xdp-tools/lib/libxdp/sharedobjs/libxdp.o build/config.o build/cmdline.o src/xdpfw.c
In file included from src/xdpfw.c:20:
modules/xdp-tools/lib/libbpf/src/libbpf.h:70:54: warning: declaration of 'enum bpf_link_type' will not be visible outside of this function [-Wvisibility]
LIBBPF_API const char *libbpf_bpf_link_type_str(enum bpf_link_type t);
^
1 warning generated.
I hope this helps and if you have any questions or run into other issues, please let me know!
XDP-Firewall# make && sudo make install mkdir -p build/ clang -O2 -c -o build/config.o src/config.c clang -O2 -c -o build/cmdline.o src/cmdline.c make -C libbpf/src make[1]: Entering directory '/root/XDP-Firewall/libbpf/src' make[1]: ../scripts/check-reallocarray.sh: Permission denied make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/root/XDP-Firewall/libbpf/src' mkdir -p build/ clang -lconfig -lelf -lz -I libbpf/src -o build/xdpfw libbpf/src/staticobjs/bpf_prog_linfo.o libbpf/src/staticobjs/bpf.o libbpf/src/staticobjs/btf_dump.o libbpf/src/staticobjs/btf.o libbpf/src/staticobjs/hashmap.o libbpf/src/staticobjs/libbpf_errno.o libbpf/src/staticobjs/libbpf_probes.o libbpf/src/staticobjs/libbpf.o libbpf/src/staticobjs/netlink.o libbpf/src/staticobjs/nlattr.o libbpf/src/staticobjs/str_error.o libbpf/src/staticobjs/xsk.o build/config.o build/cmdline.o src/xdpfw.c mkdir -p build/ clang -I libbpf/src -DBPF -O2 -emit-llvm -c -o build/xdpfw_kern.bc src/xdpfw_kern.c llc -march=bpf -filetype=obj -o build/xdpfw_kern.o build/xdpfw_kern.bc llc: error: llc: build/xdpfw_kern.bc: error: Unknown attribute kind (70) (Producer: 'LLVM13.0.1' Reader: 'LLVM 11.0.1') make: *** [Makefile:37: xdpfw_filter] Error 1
any ideas ?
@GermanKingYT Hey! It appears you have mismatched Clang and LLVM versions. Can you provide the output of the following commands please?
llc --version
clang --version
Can you also provide the Linux distro you're using and its release version/kernel? You can do that by providing the outputs of the following commands.
cat /etc/*-release
uname -a
I believe it'll be best if we started a separate issue related to this issue as well since it looks like the error you received is unrelated to the initial issue's error. I plan on closing this one soon.
i use old kernel,
clang 11 and update xdp-tools 1.4.1
but
clang -lconfig -lelf -lz -I modules/xdp-tools/lib/libbpf/src -I /usr/include -I /usr/local/include -o build/xdpfw modules/xdp-tools/lib/libbpf/src/staticobjs/bpf_prog_linfo.o modules/xdp-tools/lib/libbpf/src/staticobjs/bpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf_dump.o modules/xdp-tools/lib/libbpf/src/staticobjs/btf.o modules/xdp-tools/lib/libbpf/src/staticobjs/gen_loader.o modules/xdp-tools/lib/libbpf/src/staticobjs/hashmap.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_errno.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf_probes.o modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o modules/xdp-tools/lib/libbpf/src/staticobjs/linker.o modules/xdp-tools/lib/libbpf/src/staticobjs/netlink.o modules/xdp-tools/lib/libbpf/src/staticobjs/nlattr.o modules/xdp-tools/lib/libbpf/src/staticobjs/relo_core.o modules/xdp-tools/lib/libbpf/src/staticobjs/ringbuf.o modules/xdp-tools/lib/libbpf/src/staticobjs/str_error.o modules/xdp-tools/lib/libbpf/src/staticobjs/strset.o modules/xdp-tools/lib/libbpf/src/staticobjs/usdt.o modules/xdp-tools/lib/libxdp/sharedobjs/xsk.o modules/xdp-tools/lib/libxdp/sharedobjs/libxdp.o build/config.o build/cmdline.o src/xdpfw.c
In file included from src/xdpfw.c:19:
modules/xdp-tools/lib/libbpf/src/bpf.h:471:60: warning: declaration of 'struct bpf_link_info' will not be visible outside of this function [-Wvisibility]
LIBBPF_API int bpf_link_get_info_by_fd(int link_fd, struct bpf_link_info info, __u32 info_len);
^
In file included from src/xdpfw.c:20:
modules/xdp-tools/lib/libbpf/src/libbpf.h:70:54: warning: declaration of 'enum bpf_link_type' will not be visible outside of this function [-Wvisibility]
LIBBPF_API const char *libbpf_bpf_link_type_str(enum bpf_link_type t);
^
2 warnings generated.
/usr/bin/ld: modules/xdp-tools/lib/libbpf/src/staticobjs/libbpf.o: in function elf_find_func_offset_from_archive': /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10893: undefined reference to
zip_archive_open'
/usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10900: undefined reference to zip_archive_find_entry' /usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to
zip_archive_close'
/usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to zip_archive_close' /usr/bin/ld: /root/XDP-Firewall/modules/xdp-tools/lib/libbpf/src/libbpf.c:10934: undefined reference to
zip_archive_close'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:59: xdpfw] Error 1
how to fix? i can't change kernel but some interface issue
@namecloudz The errors you're receiving are related to ZIP functions and references missing in the version of LibBPF you're using. The latest version of LibBPF appears to compile a new zip.c file into a shared/static object while this firewall currently uses an outdated version of LibBPF that does not. Since we still use raw LibBPF functionality in the XDP Firewall loader (which I should definitely look into eliminating when I have the time so that we only rely on xdp-tools
independently), we manually link the LibBPF's static objects here. The new ZIP object is not linked here which is causing the error you're receiving most likely.
There are two options that will most likely resolve this issue.
The first option is downgrading the xdp-tools
submodule to commit 81ea740 and the libbpf
submodule (within the xdp-tools
submodule) to commit a032540 (if you reinitialize the XDP firewall repository with the --recursive
flag via git clone --recursive https://github.com/gamemann/XDP-Firewall.git
or use git submodule update --init
in a vanilla repository , it should automatically checkout these commits that compile properly for XDP Firewall).
The second option is to edit the Makefile
and add $(LIBBPFSRC)/staticobjs/zip.o
and any other new static objects to the LIBBPFOBJS
variable here so that they are linked.
I hope this helps!
Edit
I just relooked at xdp-tools
's code and wanted to note that we won't be able to rely on xdp-tools
separately since xdp-tools
doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still link libbpf
in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of the xdp-tools
submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely on xdp-tools
for all BPF functionality in the project.
I've updated the project's xdp-tools
submodule to its latest commit, cae9c91, which included linking the zip
static object from libbpf
. Try pulling the recent changes and building again. libbpf
is now on commit fbd60db within the xdp-tools
submodule.
I tested these changes in a dev environment and they appear to be working fine. The GitHub workflow I made also succeeded in building the project.
thank you,
and i will try test in 22.04
libbpf: elf: skipping unrecognized data section(7) .xdp_run_config libbpf: elf: skipping unrecognized data section(8) xdp_metadata libbpf: elf: skipping unrecognized data section(23) .eh_frame libbpf: elf: skipping relo section(24) .rel.eh_frame for section(23) .eh_frame libbpf: elf: skipping unrecognized data section(7) xdp_metadata libbpf: prog 'xdp_pass': BPF program load failed: Invalid argument libbpf: prog 'xdp_pass': failed to load: -22 libbpf: failed to load object '/usr/local/lib/bpf/xdp-dispatcher.o' libbpf: elf: skipping unrecognized data section(7) xdp_metadata libbpf: elf: skipping unrecognized data section(7) xdp_metadata libbpf: elf: skipping unrecognized data section(7) xdp_metadata Loaded XDP program on mode DRV/native.
@namecloudz You're welcome! It looks like it's working from the output you gave. Can you confirm this? If so, would it be alright to close this issue?
Edit I just relooked at
xdp-tools
's code and wanted to note that we won't be able to rely onxdp-tools
separately sincexdp-tools
doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still linklibbpf
in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of thexdp-tools
submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely onxdp-tools
for all BPF functionality in the project.
What you mean is that XDP Firewall can't be built nor use the packages version of xdp-tools at run-time?
Edit I just relooked at
xdp-tools
's code and wanted to note that we won't be able to rely onxdp-tools
separately sincexdp-tools
doesn't include the BPF functionality the XDP Firewall loader requires. So we'll have to still linklibbpf
in the XDP Firewall loader separately for now, but we do use the LibBPF library that is cloned inside of thexdp-tools
submodule. This doesn't impact the solutions above and they should still work fine, I just wanted to clarify this in the case anybody wonders why we can't rely onxdp-tools
for all BPF functionality in the project.What you mean is that XDP Firewall can't be built nor use the packages version of xdp-tools at run-time?
The firewall does use xdp-tools
, but I was initially hoping that we could eliminate the need of using LibBPF directly in the xdpfw.c
file and replacing it with functionality from xdp-tools. However, xdp-tools doesn't provide the BPF functionality we'd need from what I've seen (e.g. adding/deleting entries from a BPF map). Therefore, we still need to link the LibBPF objects here in this firewall's Makefile
.
For example, if we didn't need to link the LibBPF objects directly in our Makefile
, the errors from this reply probably wouldn't have occurred since xdp-tools would have handled linking the LibBPF objects which would have included the newer zip.o
object file.
I hope this clarifies what I meant, but if not, feel free to open a new issue (this one was closed, so I don't think you can reply)! 😄
root@filter:~/XDP-Firewall# make mkdir -p build/ clang -O2 -c -o build/config.o src/config.c clang -O2 -c -o build/cmdline.o src/cmdline.c make -C modules/xdp-tools make[1]: Entering directory '/root/XDP-Firewall/modules/xdp-tools' sh configure Found clang binary 'clang' with version 11 (from 'Ubuntu clang version 11.1.0-++20211011094159+1fdec59bffc1-1~exp1~20211011214622.5') bpftool doesn't support skeleton generation make[1]: [Makefile:49: config.mk] Error 1 make[1]: Leaving directory '/root/XDP-Firewall/modules/xdp-tools' make: [Makefile:75: libxdp] Error 2 root@filter:~/XDP-Firewall# make libxdp make -C modules/xdp-tools make[1]: Entering directory '/root/XDP-Firewall/modules/xdp-tools' sh configure Found clang binary 'clang' with version 11 (from 'Ubuntu clang version 11.1.0-++20211011094159+1fdec59bffc1-1~exp1~20211011214622.5') bpftool doesn't support skeleton generation make[1]: [Makefile:49: config.mk] Error 1 make[1]: Leaving directory '/root/XDP-Firewall/modules/xdp-tools' make: [Makefile:75: libxdp] Error 2 root@filter:~/XDP-Firewall#