search

chainguard-dev / bincapz

detect malicious program behaviors
Apache License 2.0
387 stars 24 forks source link

Add support for archives within directories #174

Closed egibs closed 2 months ago

egibs commented 2 months ago

Relates to: #173

Previously, only single archives were supported. This PR adds support for any number of archive files encountered during a recursive scan.

Since recursiveScan became unwieldy with the changes introduced in this PR, I split out the archive and file handling into separate functions.

To handle archives within directories, the processArchive function recursively scans the new temporary directory created via the archive function.

Example --

❯ ls ~/Downloads/apko_tar_gzs/
Permissions Size User  Date Modified Name
.rw-r--r--@  14M egibs 30 Apr 07:42  apko_0.13.2_darwin_amd64.tar.gz
.rw-r--r--@  14M egibs 30 Apr 07:42  apko_0.13.2_darwin_arm64.tar.gz
.rw-r--r--@  14M egibs 30 Apr 07:42  apko_0.13.2_linux_386.tar.gz
.rw-r--r--@  15M egibs 30 Apr 07:42  apko_0.13.2_linux_amd64.tar.gz
.rw-r--r--@  14M egibs 30 Apr 07:42  apko_0.13.2_linux_arm64.tar.gz
❯ go run . --format simple ~/Downloads/apko_tar_gzs
#
# /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/bincapz-apko_0.13.2_darwin_amd64.tar.gz1809843093/apko_0.13.2_darwin_amd64/apko
archives/zip
combo/dropper/bash
combo/stealer/ssh
compression/bzip2
compression/gzip
compression/zstd
crypto/aes
crypto/ecdsa
crypto/ed25519
crypto/tls
data/embedded/pem/certificate
data/embedded/pem/test_key
data/embedded/ssh/signature
data/embedded/zstd
encoding/base64
encoding/json
encoding/json/decode
encoding/json/encode
env/HOME
env/USER
evasion/content/length/0
exec/program
fs/blkid
fs/directory/create
fs/directory/list
fs/directory/remove
fs/fifo/create
fs/file/delete
fs/file/read
fs/file/stat
fs/file/truncate
fs/link/create
fs/link/read
fs/lock/update
fs/mount
fs/node/create
fs/permission/chown
fs/permission/modify
fs/swap/off
fs/swap/on
fs/symlink/resolve
fs/tempfile/create
fs/unmount
hash/blake2b
kernel/cpu/info
kernel/pivot_root
kernel/ptrace
kernel/uname/get
net/dns
net/dns/reverse
net/dns/txt
net/download
net/fetch
net/hostname/resolve
net/hostport/parse
net/http/accept/encoding
net/http/auth
net/http/cookies
net/http/post
net/http/request
net/http2
net/http_proxy
net/interface/list
net/ip
net/ip/multicast/send
net/ip/parse
net/mac/address
net/sendfile
net/socket/connect
net/socket/listen
net/socket/local/address
net/socket/peer/address
net/socket/receive
net/socket/send
net/socks5
net/ssh
net/stat
net/udp/receive
net/udp/send
net/upload
net/url
net/url/encode
net/url/request
process/chroot
process/create
process/find
process/multithreaded
process/unshare
process/username/get
ref/path/bin/su
ref/path/etc
ref/path/etc/hosts
ref/path/etc/resolv.conf
ref/path/hidden
ref/path/home
ref/path/home/config
ref/path/home_library
ref/path/relative
ref/path/root
ref/path/usr/bin
ref/path/usr/local
ref/path/usr/sbin
ref/path/var
ref/site/url
ref/words/password
ref/words/server_address
secrets/keychain
secrets/private_key
secrets/ssh
security_controls/linux/selinux
shell/background/sleep
shell/exec
time/clock/set
#
#
# /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/bincapz-apko_0.13.2_darwin_arm64.tar.gz4157572039/apko_0.13.2_darwin_arm64/apko
archives/zip
combo/dropper/bash
combo/stealer/ssh
compression/bzip2
compression/gzip
compression/zstd
crypto/aes
crypto/ecdsa
crypto/ed25519
crypto/tls
data/embedded/pem/certificate
data/embedded/pem/test_key
data/embedded/ssh/signature
data/embedded/zstd
encoding/base64
encoding/json
encoding/json/decode
encoding/json/encode
env/HOME
env/USER
evasion/content/length/0
exec/program
fs/blkid
fs/directory/create
fs/directory/list
fs/directory/remove
fs/fifo/create
fs/file/delete
fs/file/read
fs/file/stat
fs/file/truncate
fs/link/create
fs/link/read
fs/lock/update
fs/mount
fs/node/create
fs/permission/chown
fs/permission/modify
fs/swap/off
fs/swap/on
fs/symlink/resolve
fs/tempfile/create
fs/unmount
hash/blake2b
kernel/cpu/info
kernel/pivot_root
kernel/ptrace
kernel/uname/get
net/bpf
net/dns
net/dns/reverse
net/dns/txt
net/download
net/fetch
net/hostname/resolve
net/hostport/parse
net/http/accept/encoding
net/http/auth
net/http/cookies
net/http/post
net/http/request
net/http2
net/http_proxy
net/interface/list
net/ip
net/ip/multicast/send
net/ip/parse
net/mac/address
net/sendfile
net/socket/connect
net/socket/listen
net/socket/local/address
net/socket/peer/address
net/socket/receive
net/socket/send
net/socks5
net/ssh
net/stat
net/udp/receive
net/udp/send
net/upload
net/url
net/url/encode
net/url/request
process/chdir
process/chroot
process/create
process/find
process/multithreaded
process/unshare
process/username/get
ref/path/bin/su
ref/path/etc
ref/path/etc/hosts
ref/path/etc/resolv.conf
ref/path/hidden
ref/path/home
ref/path/home/config
ref/path/home_library
ref/path/relative
ref/path/root
ref/path/usr/bin
ref/path/usr/local
ref/path/usr/sbin
ref/path/var
ref/site/url
ref/words/password
ref/words/server_address
secrets/keychain
secrets/private_key
secrets/ssh
security_controls/linux/selinux
security_controls/linux/ufw
shell/background/sleep
shell/exec
time/clock/set
#
#
# /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/bincapz-apko_0.13.2_linux_386.tar.gz3595171521/apko_0.13.2_linux_386/apko
archives/zip
combo/dropper/bash
combo/stealer/ssh
compression/bzip2
compression/gzip
compression/zstd
crypto/aes
crypto/ecdsa
crypto/ed25519
crypto/tls
data/embedded/pem/certificate
data/embedded/pem/test_key
data/embedded/ssh/signature
data/embedded/zstd
encoding/base64
encoding/json
encoding/json/decode
encoding/json/encode
env/HOME
env/USER
evasion/content/length/0
exec/program
fs/blkid
fs/directory/create
fs/directory/list
fs/directory/remove
fs/fifo/create
fs/file/delete
fs/file/read
fs/file/stat
fs/link/create
fs/link/read
fs/lock/update
fs/mount
fs/node/create
fs/permission/chown
fs/permission/modify
fs/swap/off
fs/swap/on
fs/symlink/resolve
fs/tempfile/create
fs/unmount
hash/blake2b
kernel/cpu/info
kernel/hostname/get
kernel/netlink
kernel/pivot_root
kernel/uname/get
net/dns
net/dns/reverse
net/dns/txt
net/download
net/fetch
net/hostname/resolve
net/http/accept/encoding
net/http/auth
net/http/cookies
net/http/post
net/http/request
net/http2
net/http_proxy
net/interface/list
net/ip
net/ip/parse
net/mac/address
net/sendfile
net/socket/listen
net/socket/local/address
net/socket/peer/address
net/socket/receive
net/socket/send
net/socks5
net/ssh
net/stat
net/udp/receive
net/udp/send
net/upload
net/url
net/url/encode
net/url/request
process/chroot
process/find
process/groups/set
process/unshare
process/username/get
ref/path/bin/su
ref/path/etc
ref/path/etc/hosts
ref/path/etc/resolv.conf
ref/path/hidden
ref/path/home
ref/path/home/config
ref/path/relative
ref/path/root
ref/path/usr/bin
ref/path/usr/local
ref/path/usr/sbin
ref/path/var
ref/site/url
ref/words/password
ref/words/server_address
secrets/keychain
secrets/private_key
secrets/ssh
security_controls/linux/selinux
security_controls/linux/ufw
shell/background/sleep
shell/exec
time/clock/set
#
#
# /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/bincapz-apko_0.13.2_linux_amd64.tar.gz358117296/apko_0.13.2_linux_amd64/apko
archives/zip
combo/dropper/bash
combo/stealer/ssh
compression/bzip2
compression/gzip
compression/zstd
crypto/aes
crypto/ecdsa
crypto/ed25519
crypto/tls
data/embedded/pem/certificate
data/embedded/pem/test_key
data/embedded/ssh/signature
data/embedded/zstd
encoding/base64
encoding/json
encoding/json/decode
encoding/json/encode
env/HOME
env/USER
evasion/content/length/0
exec/program
fs/blkid
fs/directory/create
fs/directory/list
fs/directory/remove
fs/fifo/create
fs/file/delete
fs/file/read
fs/file/stat
fs/link/create
fs/link/read
fs/lock/update
fs/mount
fs/node/create
fs/permission/chown
fs/permission/modify
fs/swap/off
fs/swap/on
fs/symlink/resolve
fs/tempfile/create
fs/unmount
hash/blake2b
kernel/cpu/info
kernel/hostname/get
kernel/netlink
kernel/pivot_root
kernel/uname/get
net/dns
net/dns/reverse
net/dns/txt
net/download
net/fetch
net/hostname/resolve
net/http/accept/encoding
net/http/auth
net/http/cookies
net/http/post
net/http/request
net/http2
net/http_proxy
net/interface/list
net/ip
net/ip/parse
net/mac/address
net/sendfile
net/socket/listen
net/socket/local/address
net/socket/peer/address
net/socket/receive
net/socket/send
net/socks5
net/ssh
net/stat
net/udp/receive
net/udp/send
net/upload
net/url
net/url/encode
net/url/request
process/chroot
process/find
process/unshare
process/username/get
ref/path/bin/su
ref/path/etc
ref/path/etc/hosts
ref/path/etc/resolv.conf
ref/path/hidden
ref/path/home
ref/path/home/config
ref/path/relative
ref/path/root
ref/path/usr/bin
ref/path/usr/local
ref/path/usr/sbin
ref/path/var
ref/site/url
ref/words/password
ref/words/server_address
secrets/keychain
secrets/private_key
secrets/ssh
security_controls/linux/selinux
shell/background/sleep
shell/exec
time/clock/set
#
#
# /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/bincapz-apko_0.13.2_linux_arm64.tar.gz1015874883/apko_0.13.2_linux_arm64/apko
archives/zip
combo/dropper/bash
combo/stealer/ssh
compression/bzip2
compression/gzip
compression/zstd
crypto/aes
crypto/ecdsa
crypto/ed25519
crypto/tls
data/embedded/pem/certificate
data/embedded/pem/test_key
data/embedded/ssh/signature
data/embedded/zstd
encoding/base64
encoding/json
encoding/json/decode
encoding/json/encode
env/HOME
env/USER
evasion/content/length/0
exec/program
fs/blkid
fs/directory/create
fs/directory/list
fs/directory/remove
fs/fifo/create
fs/file/delete
fs/file/read
fs/file/stat
fs/link/create
fs/link/read
fs/lock/update
fs/mount
fs/node/create
fs/permission/chown
fs/permission/modify
fs/swap/off
fs/swap/on
fs/symlink/resolve
fs/tempfile/create
fs/unmount
hash/blake2b
kernel/cpu/info
kernel/hostname/get
kernel/netlink
kernel/pivot_root
kernel/uname/get
net/dns
net/dns/reverse
net/dns/txt
net/download
net/fetch
net/hostname/resolve
net/http/accept/encoding
net/http/auth
net/http/cookies
net/http/post
net/http/request
net/http2
net/http_proxy
net/interface/list
net/ip
net/ip/parse
net/mac/address
net/sendfile
net/socket/listen
net/socket/local/address
net/socket/peer/address
net/socket/receive
net/socket/send
net/socks5
net/ssh
net/stat
net/udp/receive
net/udp/send
net/upload
net/url
net/url/encode
net/url/request
process/chdir
process/chroot
process/find
process/groups/set
process/unshare
process/username/get
ref/path/bin/su
ref/path/etc
ref/path/etc/hosts
ref/path/etc/resolv.conf
ref/path/hidden
ref/path/home
ref/path/home/config
ref/path/relative
ref/path/root
ref/path/usr/bin
ref/path/usr/local
ref/path/usr/sbin
ref/path/var
ref/site/url
ref/words/password
ref/words/server_address
secrets/keychain
secrets/private_key
secrets/ssh
security_controls/linux/selinux
shell/background/sleep
shell/exec
time/clock/set
#

make test still passes as expected with these changes and scanning single binaries and archives works as well.

cc: @tstromberg @vaikas

tstromberg commented 2 months ago

This looks good.

The next improvement that I can envision is for us to be able to present the original paths instead of the temp file path in the output. I'll open a separate issue for that.

vaikas commented 2 months ago

Looks like a merge conflict, @egibs can you resolve so that we can get this merged?

egibs commented 2 months ago

Looks like a merge conflict, @egibs can you resolve so that we can get this merged?

Done!