eBPF program is compatible only with kernel 5.x

[evilsocket]

hi, i was checking how the eBPF program gets compiled and if I'm not mistake it gets compiled with the 5.8 kernel as per instructions. doesn't this cause the output object file to be only compatible with 5.x kernels? also why compiling it this way?

also I found that the Makefile might be overcomplicated, you only need the linux-headers, and you can compile like this (example Makefile I'm using for another project):

LINUX_HEADERS_ROOT=/usr/src/linux-headers-$(shell uname -r)
ARCH_UNAME := $(shell uname -m)
ARCH ?= $(ARCH_UNAME:aarch64=arm64)

all: somename.o

linux_arch := $(ARCH:x86_64=x86)
%.o: %.c
    @echo "compiling $< to $@ ..."
    @clang \
    -D__KERNEL__ \
    -D__BPF_TRACING__ \
    -D__TARGET_ARCH_$(linux_arch) \
    -O2 -target bpf \
    -Wno-address-of-packed-member \
    -Wno-compare-distinct-pointer-types \
    -Wno-deprecated-declarations \
    -Wno-gnu-variable-sized-type-not-at-end \
    -Wno-pointer-sign \
    -Wno-pragma-once-outside-header \
    -Wno-unknown-warning-option \
    -Wno-unused-value \
    -Wunused \
    -Wall \
    -fno-stack-protector \
    -fno-jump-tables \
    -fno-unwind-tables \
    -fno-asynchronous-unwind-tables \
    -xc \
    -I . \
    -I /usr/include/${ARCH}-linux-gnu/ \
    -I ${LINUX_HEADERS_ROOT}/include \
    -I ${LINUX_HEADERS_ROOT}/include/uapi \
    -I ${LINUX_HEADERS_ROOT}/include/generated \
    -I ${LINUX_HEADERS_ROOT}/include/generated/uapi \
    -I ${LINUX_HEADERS_ROOT}/tools/testing/selftests \
    -I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include \
    -I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include/generated \
    -I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include/generated/uapi \
    -c $< -o $@

this will remove the linux-source compilation dependency and all the other steps, but it will still generate an object file that will only be compatible with the targeted kernel ... or am I missing something?

cc @gustavo-iniguez-goya @themighty1

[evilsocket]

oh also by doing this, you don't need to patch the headers

[themighty1]

@evilsocket , 5.8 is just a kernel number that I chose. Might have been any other kernel number. The ebpf program needs to know the offsets of kernel's tcp/udp structs. They have been unchanged since at least 4.4. So compiling for 5.8 will create an ebpf prog which works on older kernels too.

I just took the Makefile from the kernel tree. There is definitely room for slimming it down, thanks.

[themighty1]

And yes, I tested the resulting .o file on many kernels from 4.4 up to the most recent 5.*, it ran fine.

[evilsocket]

@themighty1 the problem is not only the offsets of the structs, but also that the various bpf_* functions might be mapped to different opcodes ... i'm working a bit with ebpf recently and i think the only solution is installing all the kernel headers for the versions we want to support (fortunately just the headers) and precompile object files for each one ... otherwise we need to bring in llvm/clang/bcc dependency and compile the program on the host directly, that would produce a working object file in any ebpf supported kernel

[evilsocket]

i'm talking just out of theoretical knowledge, maybe i'm wrong? did you try to run an object file compiled with 5.x on a 4.x?

[themighty1]

Just to clarify: I compiled the .o file against 5.8 kernel and took that .o file to 4.4 kernel, and it ran fine.

[evilsocket]

wow ... that is an extremely good yet unexpected news, it goes against everything i've been reading so far about eBPF XD or maybe your compilation method does the trick?

[gustavo-iniguez-goya]

I also tested it on several kernels: bpf .o object tested successfully on kernels 4.15.0, 5.4.0, 5.8.4 and 5.10.0 (and 4.19 (i386).

https://github.com/evilsocket/opensnitch/pull/397#issuecomment-813053019

[evilsocket]

sorry guys i also tested and it works (also across different architectures!) ... for some reason i assumed there was a strict dependency on the kernel major version. thanks!

[gustavo-iniguez-goya]

np! by the way, on which architectures? I haven't tested it properly on arm*

[evilsocket]

@gustavo-iniguez-goya i just compiled the object files of another project (with kernel headers only) on 5.8.0-53-generic (x86_64) and tested them on 5.10.17-v7+ (armv7l) ... from my desktop to a raspberry pi basically