Describe the bug
Pixie requires linux kernel headers since BCC is used to compile our eBPF programs. These headers can either be supplied by installing the distro provided linux headers package or installed by pixie through our prepackaged linux headers.
When working with @feynmanliang, we realized that installing an openSUSE MicroOS linux headers package results in Pixie falling back to our prepackaged headers. While this should in theory result in equivalent behavior, the upstream distro's headers are preferred since they are more likely to be what the given kernel expects.
The openSUSE linux headers use a relative symlink and Pixie's logic to locate the headers fails for this case:
# File layout seen after installing openSUSE headers
# ls -alh /lib/modules/6.11.0-1-default/
total 43M
drwxr-xr-x. 1 root root 674 Oct 2 18:10 .
drwxr-xr-x. 1 root root 32 Sep 28 00:36 ..
-rw-r--r--. 1 root root 65 Sep 25 05:56 .vmlinuz.hmac
-rw-r--r--. 1 root root 7.6M Sep 25 05:56 System.map
lrwxrwxrwx. 1 root root 46 Sep 25 05:56 build -> ../../../src/linux-6.11.0-1-obj/x86_64/default
# PEM logs that show installed headers are misdetected as missing
I20241002 18:32:19.083017 22431 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.11.0-1-default
I20241002 18:32:19.083026 22431 linux_headers.cc:381] Detected kernel release (uname -r): 6.11.0-1-default
I20241002 18:32:19.083271 22431 linux_headers.cc:396] Not Found : Could not find 'source' or 'build' under /lib/modules/6.11.0-1-default.
I20241002 18:32:19.083294 22431 linux_headers.cc:244] Looking for host Linux headers at /host/lib/modules/6.11.0-1-default/build.
I20241002 18:32:19.083403 22431 linux_headers.cc:399] Not Found : Did not find host headers at resolved path: /host/../../../src/linux-6.11.0-1-obj/arm64/default.
I20241002 18:32:19.083415 22431 linux_headers.cc:342] Attempting to install packaged headers.
This is in contrast to Ubuntu's header package, which uses an absolute symlink:
$ ls -alh /lib/modules/$(uname -r) | grep build
lrwxrwxrwx 1 root root 37 Aug 8 21:15 build -> /usr/src/linux-headers-6.8.0-1013-gcp
To Reproduce
Steps to reproduce the behavior:
Using a Ubuntu instance, install the linux-headers-$(uname -r) package
Run stirling_wrapper and see that Pixie locates the headers properly
Replace /lib/modules/$(uname -r)/build with an openSUSE style, relative symlink
Describe the bug Pixie requires linux kernel headers since BCC is used to compile our eBPF programs. These headers can either be supplied by installing the distro provided linux headers package or installed by pixie through our prepackaged linux headers.
When working with @feynmanliang, we realized that installing an openSUSE MicroOS linux headers package results in Pixie falling back to our prepackaged headers. While this should in theory result in equivalent behavior, the upstream distro's headers are preferred since they are more likely to be what the given kernel expects.
The openSUSE linux headers use a relative symlink and Pixie's logic to locate the headers fails for this case:
This is in contrast to Ubuntu's header package, which uses an absolute symlink:
To Reproduce Steps to reproduce the behavior:
linux-headers-$(uname -r)packagestirling_wrapperand see that Pixie locates the headers properly/lib/modules/$(uname -r)/buildwith an openSUSE style, relative symlinkstirling_wrapperfalls back to prepackaged headers.Expected behavior Upstream installed linux headers should be correctly identified.
App information (please complete the following information):