Investigate the need for `CAP_SYS_ADMIN` in a few environments

[rafaelroquetto]

In some environments (e.g. virtual machine with Kind) require also the CAP_SYS_ADMIN capabilities. We are currently not testing for this capability and in these cases, Beyla may fail after the capabilities test with a less tailored error message. We need to investigate when CAP_SYS_ADMIN is used, and since this is a sort of "umbrella" capability, perhaps check if there are more granular capabilities that could be used in these instances instead.

[marevers]

In my environment (AKS v1.30.3) it seems I cannot make do without CAP_SYS_ADMIN. I've tried combinations of other capabilities, like this one:

        capabilities:
          add:
            - BPF
            - SYS_PTRACE
            - NET_RAW
            - CHECKPOINT_RESTORE
            - DAC_READ_SEARCH
            - PERFMON
            - SYS_RESOURCE
            - NET_ADMIN
          drop:
            - ALL

Without adding SYS_ADMIN as well, the following error is shown in the Beyla logs:

time=2024-09-12T09:37:49.867Z level=ERROR msg="Unable to load eBPF watcher for process events" component=discover.ProcessWatcher interval=5s error="instrumenting function \"sys_bind\": setting kprobe: creating perf_kprobe PMU (arch-specific fallback for \"sys_bind\"): token sys_bind: opening perf event: permission denied"

I've investigated the Linux man for more information on what CAP_SYS_ADMIN is covering, but based on the error message I cannot really determine what of the mentioned actions is triggering the error.

Other environments I've tested before (EKS, versions between 1.27 and 1.29) work fine without SYS_ADMIN.

[grcevski]

The finer grained permissions work only if:

• Linux Kernel >= 5.8
• cri-o >= v1.22.0 or containerd >= v1.5.0

So it's dependent on the Linux kernel version as well as the container runtime version. If those conditions above are not met CAP_SYS_ADMIN is always required.

[marevers]

@grcevski interestingly enough the AKS v1.30.3 environment has the following versions:

• Kernel 5.15.0-1068-azure
• Containerd 1.7.20-1

So based on your parameters it should work without CAP_SYS_ADMIN, but it still complains. There might be other requirements as well, or perhaps this Azure-specific kernel version works a bit differently than the default one.