TestSimple/linux/clean/busybox: failing due to overridden "none" results

[tstromberg]

It looks like our tests are adding none results to the simple output for overridden rules, unlike make refresh-sample-testdata:

    --- FAIL: TestSimple/linux/clean/busybox (0.08s)
        samples_test.go:160: Simple output mismatch: (-want +got):
              (
                """
                ... // 63 identical lines
                fs/watch: low
                impact/ddos/raw_flooder: medium
            +   impact/degrade/infection: none
            +   impact/degrade/linux_paths: none
            +   impact/remote_access/reverse_shell: none
                lateral/scan/tool: medium
                net/ip/addr: medium
                ... // 24 identical lines
                """
              )

Example: https://github.com/chainguard-dev/malcontent/actions/runs/11676935450/job/32514021648?pr=584#step:8:420

Blocks #584

[egibs]

If I scan busybox locally I don't see any behaviors with none for their severity:

go run cmd/mal/mal.go --format simple analyze ./out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534/linux/clean/busybox
# out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534/linux/clean/busybox: medium
c2/addr/ip_port: medium
credential/os/gshadow: medium
credential/os/shadow: medium
credential/password: low
data/compression/bzip2: low
data/compression/gzip: low
data/encoding/base64: low
data/random/insecure: low
discover/group/lookup: medium
discover/network/interface_get: low
discover/network/netstat: medium
discover/processes/pgrep: medium
discover/system/cpu_info: low
discover/system/platform: low
discover/system/sysinfo: medium
discover/user/HOME: low
discover/user/USER: low
discover/user/name_get: medium
evasion/hide_artifacts/pivot_root: medium
evasion/hijack_execution/LD_LIBRARY_PATH: low
evasion/logging/current_logins: medium
evasion/process_injection/ptrace: medium
exec/conditional/LANG: low
exec/program: medium
exec/program/background: low
exec/reconfigure/hostname_set: low
exec/shell/SHELL: low
exec/shell/TERM: low
exec/shell/exec: medium
exec/tty/parameters_get: medium
exec/tty/pathname: medium
false-positives/busybox: low
fs/directory/create: low
fs/directory/remove: low
fs/fifo_create: low
fs/file/copy: medium
fs/file/delete: low
fs/file/times_set: medium
fs/file/truncate: low
fs/link_read: low
fs/lock_update: low
fs/mount: low
fs/mounts_read: medium
fs/node_create: low
fs/path/etc: low
fs/path/home: medium
fs/path/tmp: medium
fs/path/usr_bin: low
fs/path/usr_sbin: low
fs/path/var: low
fs/path/var_log: medium
fs/permission/chown: low
fs/permission/modify: medium
fs/proc/arbitrary_pid: medium
fs/proc/meminfo: medium
fs/proc/mounts: medium
fs/proc/stat: medium
fs/symlink_resolve: low
fs/tempdir: low
fs/tempdir/TMPDIR: low
fs/tempdir/create: low
fs/tempdir/tempfile_create: low
fs/watch: low
impact/ddos/raw_flooder: medium
lateral/scan/tool: medium
net/ip/addr: medium
net/ip/icmp: medium
net/ip/multicast_send: low
net/ip/parse: medium
net/ip/string: medium
net/ip/tcp_state_tracker: medium
net/resolve/hostname: low
net/resolve/hostport_parse: low
net/socket/local_addr: low
net/socket/raw_s: medium
net/socket/receive: low
net/socket/send: low
net/url/embedded: low
os/time/clock_set: low
persist/daemon: medium
process/chdir: low
process/chroot: low
process/create: low
process/executable_path: low
process/groupid_set: low
process/groups_set: low
process/namespace_set: low
process/parent_pid_get: low
process/userid_set: low

Test looks good too:

PASS
ok      github.com/chainguard-dev/malcontent/out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534   226.874s

⏵ Task `go test ./out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534 -run TestSimple` finished successfully
⏵ Command: /usr/local/bin/fish -i -c 'go test -run ^TestSimple\$'

I'll see if there's anything in #584 that would be a contributing factor.

[egibs]

Scanning busybox from your branch also doesn't show those findings. 🧐

[egibs]

Ah, okay. We're setting the override severities to ignore. I'll fix this up.

To catch this (and the offending rules) I had to run:

$ go run cmd/mal/mal.go --format strings --min-risk any --ignore-tags harmless analyze ./out/samples-e58368a24b930f7dcf555678a8bc63f9d45aef24/linux/clean/busybox