open-telemetry / opentelemetry-ebpf-profiler

The production-scale datacenter profiler (C/C++, Go, Rust, Python, Java, NodeJS, .NET, PHP, Ruby, Perl, ...)
Apache License 2.0
2.32k stars 245 forks source link

Issue with some kernel modules #157

Closed mchataigner closed 5 days ago

mchataigner commented 1 week ago

Hello, I’m trying to run the agent on my server but it crashes during startup. The problem seems to come from one kernel module not being mapped correctly:

Failed to load eBPF tracer: failed to read kernel modules: addresses from modules is zero - check process permissions: 'nls_ucs2_utils 8192 1 cifs, Live 0x0000000000000000' 

and indeed, if I have a look at the /proc/modules file as root, I can see all modules having a valid address and this module nls_ucs2_utils having address 0x0000000000000000.

My understanding is that this check is here for permission validation, could it be possible not to fail if only few modules have address 0x0 but the remaining modules having a valid address?

christos68k commented 1 week ago

Hi Mathieu, can you execute cat /sys/module/nls_ucs2_utils/sections/.text and cat /proc/sys/kernel/kptr_restrict as root? Also, what Linux distribution and kernel version is your server running?

mchataigner commented 1 week ago

Sure, it’s a proxmox server (using Debian 12) with fairly recent kernel 6.8.12

# cat /sys/module/nls_ucs2_utils/sections/.text
cat: /sys/module/nls_ucs2_utils/sections/.text: No such file or directory
# cat /proc/sys/kernel/kptr_restrict
0
# ls -la /sys/module/nls_ucs2_utils/sections/
total 0
drwxr-xr-x 2 root root  0 Sep 16 13:01 .
drwxr-xr-x 5 root root  0 Sep 14 14:04 ..
-r-------- 1 root root 19 Sep 16 13:01 .data
-r-------- 1 root root 19 Sep 16 13:01 .gnu.linkonce.this_module
-r-------- 1 root root 19 Sep 16 13:01 __kcrctab_gpl
-r-------- 1 root root 19 Sep 16 13:01 __ksymtab_gpl
-r-------- 1 root root 19 Sep 16 13:01 __ksymtab_strings
-r-------- 1 root root 19 Sep 16 13:01 .note.gnu.build-id
-r-------- 1 root root 19 Sep 16 13:01 .note.Linux
-r-------- 1 root root 19 Sep 16 13:01 .rodata
-r-------- 1 root root 19 Sep 16 13:01 .strtab
-r-------- 1 root root 19 Sep 16 13:01 .symtab
# uname -a
Linux homelab 6.8.12-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.12-1 (2024-08-05T16:17Z) x86_64 GNU/Linux
# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
fabled commented 1 week ago

Ok, seems this is a module without any code, but data only. Perhaps update the PR to mention this corner case.