tstromberg
commented
4 years ago /cc @priyawadhwa
Open alban opened 4 years ago
tstromberg
commented
4 years ago /cc @priyawadhwa
tstromberg
commented
4 years ago @alban - building the headers into the kernel is likely the nicer of the two options, but I could go either way.
I'd be happy to approve a PR that does this: https://minikube.sigs.k8s.io/docs/contrib/building/iso/#adding-kernel-modules
priyawadhwa
commented
4 years ago @alban we have a doc on running BCC tools as a container in minikube, maybe this would work for you until the headers get added to the kernel
https://minikube.sigs.k8s.io/docs/tutorials/ebpf_tools_in_minikube/
alban
commented
4 years ago This issue should be reopened:
Minikube v1.12.0-beta.1 had Linux 5.4.40 (with CONFIG_IKHEADERS) but Minikube v1.12.0 has Linux 4.19.107 (without CONFIG_IKHEADERS since this build option only appeared in Linux 5.2).
See the revert commit: https://github.com/kubernetes/minikube/pull/8649
fejta-bot
commented
3 years ago Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
alban
commented
3 years ago /remove-lifecycle stale
fejta-bot
commented
3 years ago Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale
mauriciovasquezbernal
commented
3 years ago /remove-lifecycle stale
alban
commented
3 years ago Additionally, I would like to have CONFIG_DEBUG_INFO_BTF (available from Linux 5.2) to benefit from BPF programs compiled with CO-RE.
fejta-bot
commented
3 years ago Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale
mauriciovasquezbernal
commented
3 years ago /remove-lifecycle stale
sharifelgamal
commented
3 years ago So this won't be able to be supported until we update the kernel version of the ISO to v5, which we currently cannot for several reasons. I'm freezing this so that it won't get closed while we wait for that.
pnasrat
commented
11 months ago I came across this issue while experimenting with eBPF and minikube and it looks like current Minikube has the required support as now on the 5.10 kernel per above mentioned #12707 and this issue may be closable (bar the doc updated mentioned below)
eBPF tools can be shown to work in a number of ways by eBPF tools that support the options requested
Note the blog post Running eBPF Tools in Minikube would need to be updated.
/sys/kernel/kheaders.tar.xz, /sys/fs/bpf'zlim/bcc:latest uses libbcc Version: 0.8.0-78.git.eba6beb4 from Feb 2019 so doesn't include the necessary to load from https://github.com/iovisor/bcc/pull/2312 - below I've just used a fedora container and installed bcc into it to testminikube version
minikube version: v1.31.2
commit: fd7ecd9c4599bef9f04c0986c4a0187f98a4396e
minikube ssh -- uname -r
5.10.57This means that if a container has mounted /sys/kernel/kheaders.tar.xz and is privileged with the right tools installed (xz, bcc) you can run execsnoop. Note needs xz here to read headers
minikube ssh
docker run --rm --privileged -v /sys/kernel/kheaders.tar.xz:/sys/kernel/kheaders.tar.xz -it -v /lib/modules:/lib/modules:ro -v /sys/fs/bpf:/sys/fs/bpf fedora:latest /bin/bash
dnf install -y xz bcc
cd /usr/share/bcc/tools
./execsnoop
[root@15ccc8e21c91 tools]# ./execsnoop
PCOMM PID PPID RET ARGS
bridge 103806 1041 0 /opt/cni/bin/bridge
portmap 103811 1041 0 /opt/cni/bin/portmap
iptables 103815 2295 0 /usr/sbin/iptables -w 5 -W 100000 -S KUBE-KUBELET-CANARY -t mangle
ip6tables 103816 2295 0 /usr/sbin/ip6tables -w 5 -W 100000 -N KUBE-KUBELET-CANARY -t mangle
ip6tables 103817 2295 0 /usr/sbin/ip6tables -w 5 -W 100000 -N KUBE-KUBELET-CANARY -t nat
runc 103820 8279 0 /usr/bin/runc --root /var/run/docker/runtime-runc/moby --log /var/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/651ba58ea0979c86fd16e779b30e6246c85dada4d84bb5dc0ec5547db06 --log-format json exec --process /tmp/runc-process1214458527 --detach --pid-file /var/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/651ba58ea0979c86fd16e779b30e6246c85dada4d84bb5dc0ec5547db06 651ba58ea0979c86fd16e779b30e6246c85dada4d84bb5dc0ec5547db06cb964
I would like the Minikube kernel to be compiled with CONFIG_IKHEADERS.
This is missing from linux_defconfig.
BCC tools need to have access to kernel headers. It can be done either by installing
linux-headerspackages or by having a kernel compiled withCONFIG_IKHEADERSso that enough information can be retrieved via/sys/kernel/kheaders.tar.xz.BCC tools are used in Inspektor Gadget, a collection of tools for developers of Kubernetes applications. I would like Inspektor Gadget to support Minikube.
/cc @mauriciovasquezbernal