Define all supported Linux versions and their kernels. Document it.

rafaeldtinoco commented 3 years ago

BIONIC (external BTF)

$ uname -a
Linux bionic 5.4.0-77-generic #86~18.04.1-Ubuntu SMP Fri Jun 18 01:23:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo TRACEE_BTF_FILE=/tmp/btf/5.4.0-77-generic.btf ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
00:45:08:482393  1000   bash             1586    1586    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
00:45:08:482406  1000   bash             1586    1586    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
00:45:08:483401  1000   bash             1603    1603    0                execve               pathname: /bin/ls, argv: [ls --color=auto]
...

FOCAL (external BTF)

BTFHUB regex for downloading new 5.11 dbgs (to generate BTF file) needs fix.

GROOVY

$ uname -a
Linux groovy 5.8.0-63-generic #71-Ubuntu SMP Tue Jul 13 15:59:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
01:08:14:597014  1000   bash             1072    1072    0                execve               pathname: /bin/ps, argv: [ps -ef]
01:08:14:614665  1000   ps               1072    1072    0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
...

HIRSUTE

$ uname -a
Linux hirsute 5.11.0-34-generic #36-Ubuntu SMP Thu Aug 26 19:22:09 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall

SAME TEST: GOOD

IMPISH

$ uname -a
Linux impish 5.13.0-14-generic #14-Ubuntu SMP Mon Aug 2 12:43:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

SAME TEST: GOOD

UPSTREAM

$ uname -a
Linux devel 5.14.0-rc5+ #4 SMP PREEMPT Sun Aug 29 02:16:18 -03 2021 x86_64 x86_64 x86_64 GNU/Linux

SAME TEST: GOOD

ARCHLINUX

$uname -a
Linux archlinux 5.13.13-arch1-1 #1 SMP PREEMPT Thu, 26 Aug 2021 19:14:36 +0000 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
2021/08/30 01:27:23 could not open /boot/config-5.13.13-arch1-1: open /boot/config-5.13.13-arch1-1: no such file or directory

need to test a regression for /proc/config.gz caused by recent changes to libbpfgo (in my branch).

CENTOS7 (UNSUPPORTED)

$ uname -a
Linux centos7-vm 3.10.0-1160.31.1.el7.centos.plus.x86_64 #1 SMP Wed Jun 9 16:22:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
2021/08/30 01:20:45 missing kernel configuration options: [CONFIG_KPROBE_EVENTS]

CENTOS8

$ uname -a
Linux centos8-vm 4.18.0-305.7.1.el8_4.centos.plus.x86_64 #1 SMP Fri Jul 2 13:54:32 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
01:22:13:628458  1000   bash             1475    1475    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
01:22:13:628492  1000   bash             1475    1475    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
01:22:13:628506  1000   bash             1475    1475    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
01:22:13:629734  1000   bash             1477    1477    0                execve               pathname: /bin/tr, argv: [tr [:lower:] [:upper:]]
01:22:13:629779  1000   bash             1477    1477    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:22:13:629952  1000   tr               1477    1477    0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve

FEDORA31 (external btf for 5.3 kernel)

2 kernels:

kernel-core-0:5.3.7-301.fc31.x86_64

$ uname -a
Linux fedora31 5.3.7-301.fc31.x86_64 #1 SMP Mon Oct 21 19:18:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ sudo TRACEE_BTF_FILE=./5.3.7-301.fc31.x86_64.btf ./tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
BTF: bpfenv = false, btfenv = true, vmlinux = false
BPF: using embedded BPF object
BTF: using BTF file from environment: ./5.3.7-301.fc31.x86_64.btf
unpacked CO:RE bpf object file into memory
TIME             UID    COMM             PID     TID     RET              EVENT                ARGS
libbpf: Error loading BTF: Invalid argument(22)
libbpf: magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 59084
str_off: 59084
str_len: 87382
btf_total_size: 146490
Invalid string section
libbpf: Error loading .BTF into kernel: -22. BTF is optional, ignoring.
...
02:34:16:801600  1000   bash             821     821     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
02:34:16:801626  1000   bash             821     821     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
02:34:16:801653  1000   bash             821     821     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
02:34:16:802565  1000   bash             856     856     0                execve               pathname: /bin/ls, argv: [ls --color=auto]
02:34:16:802650  1000   bash             856     856     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
...

error is about loading tracee-ebpf BTF file into kernel (not our BTF file) its okay if that does not work, but the message is ugly (open a bug ?)

kernel-core-0:5.8.18-100.fc31.x86_64

$ uname -a
Linux fedora31 5.8.18-100.fc31.x86_64 #1 SMP Mon Nov 2 20:32:55 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
01:52:30:839122  1000   bash             721     721     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: clone
01:52:30:839715  1000   bash             723     723     0                execve               pathname: /bin/tr, argv: [tr [:lower:] [:upper:]]
01:52:30:839751  1000   bash             723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:839907  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:839932  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:839944  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:839968  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:839988  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: execve
01:52:30:840156  1000   tr               723     723     0                cap_capable          cap: CAP_SYS_ADMIN, syscall: mmap

FEDORA32

2 kernels:

kernel-core-0:5.6.6-300.fc32.x86_64

$ uname -a
Linux fedora32 5.6.6-300.fc32.x86_64 #1 SMP Tue Apr 21 13:44:19 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:22:57:402154  1000   bash             789     789     0                execve               pathname: /bin/tr, argv: [tr [:lower:] [:upper:]]
02:22:57:406749  1000   bash             792     792     0                execve               pathname: /bin/tr, argv: [tr [:lower:] [:upper:]]
02:23:01:116755  1000   bash             794     794     0                execve               pathname: /bin/ps, argv: [ps]
02:23:01:128952  1000   ps               794     794     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
02:23:01:129107  1000   ps               794     794     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
02:23:01:129207  1000   ps               794     794     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read

kernel-core-0:5.11.22-100.fc32.x86_64

$ uname -a
Linux fedora32 5.11.22-100.fc32.x86_64 #1 SMP Wed May 19 18:58:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:21:33:570834  1000   bash             775     775     0                execve               pathname: /bin/tr, argv: [tr [:lower:] [:upper:]]
02:21:36:159634  1000   bash             777     777     0                execve               pathname: /bin/ps, argv: [ps]
02:21:36:171319  1000   ps               777     777     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
02:21:36:171546  1000   ps               777     777     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read

FEDORA33

2 kernels:

kernel-core-0:5.8.15-301.fc33.x86_64

$ uname -a
Linux fedora33 5.8.15-301.fc33.x86_64 #1 SMP Thu Oct 15 16:58:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:18:40:090650  1000   bash             755     755     0                execve               pathname: /bin/ls, argv: [ls --color=auto]
02:18:42:491772  1000   bash             757     757     0                execve               pathname: /bin/ps, argv: [ps]
02:18:42:504265  1000   ps               757     757     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
02:18:42:504411  1000   ps               757     757     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read

kernel-core-0:5.13.12-100.fc33.x86_64

$ uname -a
Linux fedora33 5.13.12-100.fc33.x86_64 #1 SMP Wed Aug 18 20:12:01 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:15:49:291793  1000   bash             845     845     0                execve               pathname: /bin/ps, argv: [ps -ef]
02:15:49:303547  1000   ps               845     845     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read

FEDORA34

2 kernels:

kernel-core-0:5.11.12-300.fc34.x86_64

$ uname -a
Linux fedora34 5.11.12-300.fc34.x86_64 #1 SMP Wed Apr 7 16:31:13 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:11:09:002514  1000   bash             748     748     0                execve               pathname: /usr/bin/grep, argv: [grep --color=auto -q systemd /proc/1/stat]
02:11:09:004647  1000   grep             748     748     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read
02:11:09:006662  1000   bash             749     749     0                execve               pathname: /usr/bin/systemctl, argv: [systemctl list-units --state=failed --no-legend --plain]
02:11:09:024367  1000   bash             751     751     0                execve               pathname: /usr/bin/locale, argv: [/usr/bin/locale]

kernel-core-0:5.13.12-200.fc34.x86_64

$ uname -a
Linux fedora34 5.13.12-200.fc34.x86_64 #1 SMP Wed Aug 18 13:27:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ sudo ./dist/tracee-ebpf --debug --trace uid=1000 --trace pid=new --trace event=execve --trace event=cap_capable --output option:detect-syscall
...
02:00:41:874953  1000   grepconf.sh      788     788     0                execve               pathname: /usr/bin/grep, argv: [grep -qsi ^COLOR.*none /etc/GREP_COLORS]
02:00:41:877483  1000   bash             789     789     0                execve               pathname: /usr/bin/grep, argv: [grep --color=auto -q systemd /proc/1/stat]
02:00:41:878556  1000   grep             789     789     0                cap_capable          cap: CAP_SYS_PTRACE, syscall: read

rafaeldtinoco commented 3 years ago

@yanivagman and @itaysk

We currently have:

Bionic and Focal BTF files generation to be fixed (in btfhub)
Archlinux using /proc/config.gz and newer versions of libbpfgo regression (will fix it)
Ugly message about not loading BTF in 5.3 kernel for Fedora 31
Ubuntu Bionic 5.4 issue solved by: https://github.com/aquasecurity/tracee/pull/972

I believe the minimum we should support is Ubuntu Bionic, Fedora 31, CentOS 8. All of them with latest kernel releases. After we agree on the list I can document this 'supported' distros and we can add more later (together with the BTFHUB automation).

yanivagman commented 3 years ago

About

Ugly message about not loading BTF in 5.3 kernel for Fedora 31

Maybe it is related to #851?

itaysk commented 3 years ago

@rafaeldtinoco can you explain what did you mean by documenting the supported version? I don't think we should say where tracee should be run just what it requires. if you mean testing different versions then yes we should but that I think should be driven by the usercases we defined and is tracked in #1022

rafaeldtinoco commented 3 years ago

@rafaeldtinoco can you explain what did you mean by documenting the supported version? I don't think we should say where tracee should be run just what it requires. if you mean testing different versions then yes we should but that I think should be driven by the usercases we defined and is tracked in #1022

Yes, when I created this 1022 did not exist yet. I wanted to make sure we would test all available kernels for the distros we would be supporting.

My intent here was to document environments we currently test tracees functionalities and call them supported, not to limit usage to those envs only.

I can close this on behalf of issue #1022 but do note we are using cloud environments there, more than distro and kernel versions.

aquasecurity / tracee