Open jpmeijers opened 1 month ago
Hi @jpmeijers ,
Can you please check if your kernel was compiled with CONFIG_HAVE_SYSCALL_TRACEPOINTS=y ?
Same question for CONFIG_FTRACE_SYSCALLS
Possibly, this is fixed by https://github.com/parca-dev/parca-agent/pull/2990
Presumably related: https://github.com/evilsocket/opensnitch/issues/774
@jpmeijers , can you please try again with the most recent image? ghcr.io/parca-dev/parca-agent:main-48f7888d
I tried the new container version on two different hosts. A physical Dell server running Unraid, and a VM running Ubuntu 20.04. Both fail with the same error:
Failed to load eBPF tracer: failed to read kernel symbols: unable to open /proc/kallsyms: open /proc/kallsyms: no such file or directory
Unraid:
# grep FTRACE /boot/config-$(uname -r)
grep: /boot/config-6.8.12-Unraid: No such file or directory
VM + Ubuntu:
# grep FTRACE /boot/config-$(uname -r)
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
# CONFIG_FTRACE_SYSCALLS is not set
The tracepoint error has however been replaced by a warning, so this issue can be closed. I'll have to open a new one about eBPF.
parca-agent-1 | 2024/10/25 13:36:42 INFO memory is not limited, skipping package=github.com/KimMachineGun/automemlimit/memlimit
parca-agent-1 | 2024/10/25 13:36:42 INFO memory is not limited, skipping package=github.com/KimMachineGun/automemlimit/memlimit
parca-agent-1 | ooooooooo. .o. .
parca-agent-1 | `888 `Y88. .888. .o8
parca-agent-1 | 888 .d88' .oooo. oooo d8b .ooooo. .oooo. .8"888. .oooooooo .ooooo. ooo. .oo. .o888oo
parca-agent-1 | 888ooo88P' `P )88b `888""8P d88' `"Y8 `P )88b .8' `888. 888' `88b d88' `88b `888P"Y88b 888
parca-agent-1 | 888 .oP"888 888 888 .oP"888 .88ooo8888. 888 888 888ooo888 888 888 888
parca-agent-1 | 888 d8( 888 888 888 .o8 d8( 888 .8' `888. `88bod8P' 888 .o 888 888 888 .
parca-agent-1 | o888o `Y888""8o d888b `Y8bod8P' `Y888""8o o88o o8888o `8oooooo. `Y8bod8P' o888o o888o "888"
parca-agent-1 | d" YD
parca-agent-1 | "Y88888P'
parca-agent-1 |
parca-agent-1 | time="2024-10-25T13:36:42Z" level=warning msg="Failed to probe tracepoint: failed to get id for tracepoint: failed to read tracepoint ID for sys_enter_mmap: open /sys/kernel/debug/tracing/events/syscalls/sys_enter_mmap/id: no such file or directory. Parca-agent may fail to run on some kernel versions."
parca-agent-1 | time="2024-10-25T13:36:42Z" level=info msg="External labels: "
parca-agent-1 | time="2024-10-25T13:36:42Z" level=info msg="Interpreter tracers: perl,php,python,hotspot,ruby,v8,dotnet"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=info msg="no config file provided, using default config"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=info msg="Can't connect Docker client to [/var/run/docker.sock]"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=info msg="Environment variable KUBERNETES_SERVICE_HOST not set"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=error msg="Failed to load eBPF tracer: failed to read kernel symbols: unable to open /proc/kallsyms: open /proc/kallsyms: no such file or directory"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=error msg="======================= unexpected error ======================="
parca-agent-1 | time="2024-10-25T13:36:42Z" level=error msg="2024/10/25 13:36:42 INFO memory is not limited, skipping package=github.com/KimMachineGun/automemlimit/memlimit\ntime=\"2024-10-25T13:36:42Z\" level=warning msg=\"Failed to probe tracepoint: failed to get id for tracepoint: failed to read tracepoint ID for sys_enter_mmap: open /sys/kernel/debug/tracing/events/syscalls/sys_enter_mmap/id: no such file or directory. Parca-agent may fail to run on some kernel versions.\"\ntime=\"2024-10-25T13:36:42Z\" level=info msg=\"External labels: \"\ntime=\"2024-10-25T13:36:42Z\" level=info msg=\"Interpreter tracers: perl,php,python,hotspot,ruby,v8,dotnet\"\ntime=\"2024-10-25T13:36:42Z\" level=info msg=\"no config file provided, using default config\"\ntime=\"2024-10-25T13:36:42Z\" level=info msg=\"Can't connect Docker client to [/var/run/docker.sock]\"\ntime=\"2024-10-25T13:36:42Z\" level=info msg=\"Environment variable KUBERNETES_SERVICE_HOST not set\"\ntime=\"2024-10-25T13:36:42Z\" level=error msg=\"Failed to load eBPF tracer: failed to read kernel symbols: unable to open /proc/kallsyms: open /proc/kallsyms: no such file or directory\"\n"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=error msg="================================================================"
parca-agent-1 | time="2024-10-25T13:36:42Z" level=error msg="about to report error to server"
parca-agent-1 | time="2024-10-25T13:36:43Z" level=warning msg="finished callprotocolgrpcgrpc.componentclientgrpc.serviceparca.telemetry.v1alpha1.TelemetryServicegrpc.methodReportPanicgrpc.method_typeunarygrpc.start_time2024-10-25T13:36:42Zgrpc.request.deadline2024-10-25T13:38:42Zgrpc.codeUnimplementedgrpc.errorrpc error: code = Unimplemented desc = unexpected HTTP status code received from server: 404 (Not Found); transport: received unexpected content-type \"text/plain; charset=utf-8\"grpc.time_ms152.119"
parca-agent-1 | time="2024-10-25T13:36:43Z" level=error msg="failed to call ReportPanic(): rpc error: code = Unimplemented desc = unexpected HTTP status code received from server: 404 (Not Found); transport: received unexpected content-type \"text/plain; charset=utf-8\""
parca-agent-1 exited with code 1
Running parca-agent in Docker on Unraid 7.0.0-beta.2.
The container exits soon after startup due to it not finding
/sys/kernel/debug/tracing/events/syscalls/sys_enter_mmap/id
.Looking at the host, the kernel debugfs is mounted, but the syscalls directory does not exist.