kubearmor / KubeArmor

Runtime Security Enforcement System. Workload hardening/sandboxing and implementing least-permissive policies made easy leveraging LSMs (BPF-LSM, AppArmor).
https://kubearmor.io/
Apache License 2.0
1.49k stars 343 forks source link

bug: BPFLSM enforcer fails to load on newer kernels (6.8+) #1836

Closed DelusionalOptimist closed 2 months ago

DelusionalOptimist commented 2 months ago

Bug Report

General Information

To Reproduce

  1. Make sure your kernel has BPF LSM enabled and run KubeArmor

  2. Get the below error

    Aug 12 10:53:45 pingu kubearmor[402510]: 2024-08-12 10:53:45.620909        ERROR        error loading BPF LSM objects: field EnforceNetAccept: program enforce_net_accept: load program: permission denied: 2: (69) r2 = *(u16 *)(r2 +574): R2 invalid mem access 'trusted_ptr_or_null_' (7 line(s) omitted)
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/log.Err
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/log/logger.go:103
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/feeder.(*Feeder).Errf
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/feeder/feeder.go:430
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/enforcer/bpflsm.NewBPFEnforcer
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/enforcer/bpflsm/enforcer.go:103
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/enforcer.selectLsm
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/enforcer/runtimeEnforcer.go:106
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/enforcer.NewRuntimeEnforcer
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/enforcer/runtimeEnforcer.go:175
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/core.(*KubeArmorDaemon).InitRuntimeEnforcer
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/core/kubeArmor.go:291
    Aug 12 10:53:45 pingu kubearmor[402510]: github.com/kubearmor/KubeArmor/KubeArmor/core.KubeArmor
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/core/kubeArmor.go:547
    Aug 12 10:53:45 pingu kubearmor[402510]: main.main
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/work/KubeArmor/KubeArmor/KubeArmor/main.go:79
    Aug 12 10:53:45 pingu kubearmor[402510]: runtime.main
    Aug 12 10:53:45 pingu kubearmor[402510]:         /home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.9.linux-amd64/src/runtime/proc.go:267
  3. Try to load it manually

    libbpf: prog 'enforce_net_connect': BPF program load failed: Permission denied
    libbpf: prog 'enforce_net_connect': -- BEGIN PROG LOAD LOG --
    0: R1=ctx() R10=fp0
    ; LSM_NET(enforce_net_connect, _SOCKET_CONNECT); @ enforcer.bpf.c:437
    0: (79) r1 = *(u64 *)(r1 +0)
    func 'bpf_lsm_socket_connect' arg0 has btf_id 5777 type STRUCT 'socket'
    1: R1_w=trusted_ptr_socket()
    1: (79) r2 = *(u64 *)(r1 +24)         ; R1_w=trusted_ptr_socket() R2_w=trusted_ptr_or_null_sock(id=1)
    2: (69) r2 = *(u16 *)(r2 +574)
    R2 invalid mem access 'trusted_ptr_or_null_'
    processed 3 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
    -- END PROG LOAD LOG --
    libbpf: prog 'enforce_net_connect': failed to load: -13
    libbpf: failed to load object 'enforcer_bpfel.o'
    Error: failed to load object file

Expected behavior

KubeArmor should load BPF LSM enforcer.

DelusionalOptimist commented 2 months ago

cc @daemon1024 @Prateeknandle