Preserve original path for archives; add to output

egibs commented 2 months ago

Closes: https://github.com/chainguard-dev/bincapz/issues/178

When scanning archives, we only showed the temporary directory used to hold and extract the archive. This PR preserves the originally-provided path and displays it when scanning archives.

This change applies to both normal scans as well as diffs (output provided in the comments below).

Examples --

Terminal:

Scanned Path: /var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/apko_0.13.2_darwin_amd64.tar.gz4091940928/apko_0.13.2_darwin_amd64/apko [🚨 CRITICAL]
Original Path: /Users/egibs/Downloads/apko_tar_gzs/apko_0.13.2_darwin_amd64.tar.gz

---------------------------------------------------------------------------------------------------------------------------------------------------
RISK  KEY                              DESCRIPTION                                           EVIDENCE
---------------------------------------------------------------------------------------------------------------------------------------------------
LOW   compression/bzip2                Works with bzip2 files                                bzip2
LOW   compression/gzip                 works with gzip files                                 gzip
LOW   compression/zstd                 Zstandard: fast real-time compression algorithm       (�/�
                                                                                             zstd
LOW   crypto/aes                       Supports AES (Advanced Encryption Standard)           AES
                                                                                             crypto/aes
LOW   crypto/ecdsa                     Uses the Go crypto/ecdsa library                      crypto/ecdsa
LOW   crypto/ed25519                   Elliptic curve algorithm used by TLS and SSH          ed25519
LOW   crypto/tls                       tls                                                   TLS13
                                                                                             TLSVersion
                                                                                             crypto/tls
LOW   data/embedded/pem/certificate    Contains embedded PEM certificate                     -----BEGIN CERTIFICATE-----
LOW   data/embedded/pem/test_key       Contains TESTING KEY directive                        TESTING KEY-----
LOW   data/embedded/ssh/signature      Contains embedded SSH signature                       --BEGIN SSH SIGNATURE--
LOW   encoding/base64                  Supports base64 encoded strings                       base64
LOW   encoding/json                    Supports JSON encoded objects                         encoding/json
LOW   encoding/json/decode             Decodes JSON messages                                 json.Unmarshal
LOW   encoding/json/encode             encodes JSON                                          MarshalJSON
LOW   env/HOME                         Looks up the HOME directory for the current user      HOME
                                                                                             getenv
LOW   env/USER                         Looks up the USER name of the current user            USER
                                                                                             getenv
LOW   fs/blkid                         works with block device attributes                    blkid
LOW   fs/directory/create              creates directories                                   mkdir
LOW   fs/directory/list                Uses Go functions to list a directory                 .OpenDir
                                                                                             .ReadDir
LOW   fs/directory/remove              Uses libc functions to remove directories             Rmdir
                                                                                             rmdir
LOW   fs/fifo/create                   make a FIFO special file (a named pipe)               mkfifo
LOW   fs/file/delete                   deletes files                                         unlinkat
LOW   fs/file/read                     reads files                                           ioutil.ReadFile
                                                                                             os.(*File).Read
LOW   fs/file/stat                     access filesystem information                         fs.statDirEntry
LOW   fs/file/truncate                 truncate a file to a specified length                 ftruncate
LOW   fs/link/create                   May create hard file links                            _link
LOW   fs/link/read                     read value of a symbolic link                         readlink
LOW   fs/lock/update                   apply or remove an advisory lock on a file            flock
LOW   fs/mount                         mounts file systems                                   -o
                                                                                             mount
LOW   fs/node/create                   create device files                                   mknod
LOW   fs/swap/off                      stop swapping to a file/device                        swapoff
LOW   fs/swap/on                       start swapping to a file/device                       swapon
LOW   fs/symlink/resolve               resolves symbolic links                               realpath
LOW   fs/tempfile/create               Uses mktemp to create temporary files                 mktemp
                                                                                             temp file
LOW   fs/unmount                       unmount file system                                   umount
LOW   hash/blake2b                     Uses blake2b encryption algorithm                     blake2b
LOW   kernel/cpu/info                  gets number of processors                             nproc
LOW   kernel/pivot_root                change the root mount location                        pivot_root
LOW   net/dns                          Uses DNS (Domain Name Service)                        CNAMEResource
                                                                                             SetEDNS0
                                                                                             dnsmessage
LOW   net/dns/txt                      Uses DNS TXT (text) records                           TXT
                                                                                             dns
LOW   net/hostname/resolve             resolve network host name to IP address               LookupHostIP
                                                                                             net.hostLookup
LOW   net/hostport/parse               Network address and service translation               freeaddrinfo
                                                                                             getaddrinfo
LOW   net/http/accept/encoding         set HTTP response encoding format (example: gzip)     Accept-Encoding
LOW   net/http/auth                    makes HTTP requests with basic authentication         WWW-Authenticate
                                                                                             Www-Authenticate
                                                                                             www-authenticate
LOW   net/http/request                 makes HTTP requests                                   HTTP/1.
                                                                                             Referer
                                                                                             User-Agent
LOW   net/http2                        Uses the HTTP/2 protocol                              HTTP/2
LOW   net/http_proxy                   use HTTP proxy that requires authentication           Proxy-Authorization
LOW   net/ip                           access the internet                                   invalid packet
LOW   net/ip/multicast/send            send data to multiple nodes simultaneously            multicast
LOW   net/sendfile                     transfer data between file descriptors                sendfile
                                                                                             syscall.Sendfile
LOW   net/socket/listen                listen on a socket                                    accept
                                                                                             listen
                                                                                             socket
LOW   net/socket/local/address         get local address of connected socket                 getsockname
LOW   net/socket/peer/address          get peer address of connected socket                  getpeername
LOW   net/socket/receive               receive a message from a socket                       recvfrom
                                                                                             recvmsg
LOW   net/socket/send                  send a message to a socket                            sendmsg
                                                                                             sendto
LOW   net/udp/receive                  Listens for UDP responses                             ReadFromUDP
                                                                                             listenUDP
LOW   net/udp/send                     Sends UDP packets                                     DialUDP
                                                                                             WriteMsgUDP
LOW   net/url                          Handles URL strings                                   RequestURI
LOW   process/chroot                   change the location of root for the process           chroot
LOW   process/create                   create child process                                  _fork
LOW   process/multithreaded            creates pthreads                                      pthread_create
LOW   process/unshare                  disassociate parts of the process execution context   unshare
LOW   ref/path/bin/su                  Calls /bin/su                                         /bin/su
LOW   ref/path/etc                     path reference within /etc                            /etc/apache/mime.typeshpack
                                                                                             /etc/apk/keys/etc/apk/archcached
                                                                                             /etc/apk/lib/apk
                                                                                             /etc/apk/repositories/lib/apk/db/inst
                                                                                             /etc/apk/world
                                                                                             /etc/bash
                                                                                             /etc/busybox-paths.d/usr/bin/setkeyco
                                                                                             /etc/default/motd-newsformat
                                                                                             …
LOW   ref/path/etc/resolv.conf         accesses DNS resolver configuration                   /etc/resolv.conf
LOW   ref/path/home/config             path reference within ~/.config                       ~/.config/fish/completions/
LOW   ref/path/home_library            path reference within ~/Library                       /System/Library/Frameworks/CoreFoundation
                                                                                             /System/Library/Frameworks/Security
                                                                                             offset/Library/Caches is not definedwrite heap dump…
LOW   ref/path/usr/bin                 path reference within /usr/bin                        /usr/bin/ar/usr/bin/bc/usr/bin/dc/usr/bin/du/usr/bi…
                                                                                             /usr/bin/ascii/usr/bin/crc32/usr/bin/tsortVERSION_ID
                                                                                             /usr/bin/awk/usr/bin/cal/usr/bin/cmp/usr/bin/cut/us…
                                                                                             /usr/bin/basename/usr/bin/dos2unix/usr/bin/dpkg-deb…
                                                                                             /usr/bin/bc/usr/bin/dc/usr/bin/du/usr/bin/hd/usr/bi…
                                                                                             /usr/bin/beep/usr/bin/chrt/usr/bin/chvt/usr/bin/com…
                                                                                             /usr/bin/blkdiscard/usr/bin/dumpleases/usr/bin/ssl_…
                                                                                             /usr/bin/bunzip2/usr/bin/crontab/usr/bin/cryptpw/us…
                                                                                             …
LOW   ref/path/usr/sbin                path reference within /usr/sbin                       /usr/sbin/add-shell/usr/sbin/dhcprelay/usr/sbin/get…
                                                                                             /usr/sbin/addgroup/usr/sbin/chpasswd/usr/sbin/delgr…
                                                                                             /usr/sbin/adduser/usr/sbin/deluser/usr/sbin/flashcp…
                                                                                             /usr/sbin/arping/usr/sbin/chroot/usr/sbin/i2cget/us…
                                                                                             /usr/sbin/brctl/usr/sbin/crond/usr/sbin/fbset/usr/s…
                                                                                             /usr/sbin/chat/usr/sbin/dnsd/usr/sbin/ftpd/usr/sbin…
                                                                                             /usr/sbin/chpasswd/usr/sbin/delgroup/usr/sbin/fdfor…
                                                                                             /usr/sbin/chroot/usr/sbin/i2cget/usr/sbin/i2cset/us…
                                                                                             …
LOW   ref/path/var                     path reference within /var                            /var/cache%s
                                                                                             /var/cache/apk/etc/apk/worldCalculateWorldcache
                                                                                             /var/cache/miscAPKINDEX.tar.gzfetchAlpineKeyscfg.Ma…
                                                                                             /var/lib/db/sbomSPDXRef-Package-remote
                                                                                             /var/run/docker.sockopen
LOW   ref/site/url                     contains embedded HTTPS URLs                          https://GoString01234567beEfFgGvsignal
                                                                                             https://alpinelinux.org/releases.jsondid
                                                                                             https://github.com/chainguard-dev/apkocould
                                                                                             https://github.com/google/go-containerregistry/issu…
                                                                                             https://github.com/spf13/cobra/issues/1279
                                                                                             https://github.com/spf13/cobra/issues/1508
                                                                                             https://index.docker.io/v1/Path
                                                                                             https://index.docker.io/v2/library/ubuntu/tags/list
                                                                                             …
LOW   ref/words/password               references a 'password'                               IncorrectPasswordError
                                                                                             Password from
                                                                                             PasswordHashIterations
                                                                                             UserPassword
                                                                                             and password requiredreading
                                                                                             bson bytes as PasswordGODEBUG sys
                                                                                             passwordSet
                                                                                             passwordStdin
                                                                                             …
LOW   secrets/private_key              References private keys                               privateKey
                                                                                             private_key
LOW   time/clock/set                   set time via system clock                             adjtimex
MED   archives/zip                     Works with zip files                                  archive/zip
MED   combo/dropper/bash               may fetch file, make it executable, and run it        ./b
                                                                                             ./c
                                                                                             ./jb
                                                                                             ./line
                                                                                             ./pipe/docker
                                                                                             ./q
                                                                                             ./r
                                                                                             ./v
                                                                                             …
MED   combo/stealer/ssh                possible SSH stealer                                  .ssh
                                                                                             curl
                                                                                             socket
                                                                                             tar
                                                                                             wget
                                                                                             zip
MED   data/embedded/zstd               Contains compressed content in ZStandard format       (�/�
MED   evasion/content/length/0         Sets HTTP content length to zero                      Content-Length: 0
MED   exec/program                     executes external programs                            ).CombinedOutput
                                                                                             exec.(*Cmd).Run
MED   fs/permission/chown              Changes file ownership                                Chown
MED   fs/permission/modify             modifies file permissions                             Chmod
                                                                                             chmod
MED   kernel/ptrace                    trace or modify system calls                          ptrace
MED   kernel/uname/get                 system identification (uname)                         uname
MED   net/dns/reverse                  looks up the reverse hostname for an IP               .in-addr.arpa
                                                                                             ip6.arpa
MED   net/download                     download files                                        DownloadLocation
                                                                                             downloadLocation
                                                                                             to registrySkip downloading
MED   net/fetch                        Invokes curl                                          curl -H "
MED   net/http/cookies                 access HTTP resources using cookies                   Cookie
                                                                                             HTTP
MED   net/http/post                    submit content to websites                            HTTP
                                                                                             POST
                                                                                             http
MED   net/interface/list               list network interfaces                               ifconfig
MED   net/ip/parse                     parses IP address (IPv4 or IPv6)                      IsLinkLocalUnicast
                                                                                             IsSingleIP
MED   net/mac/address                  Retrieves network MAC address                         MAC address
MED   net/socket/connect               initiate a connection on a socket                     _connect
MED   net/socks5                       Supports SOCK5 proxies                                SOCKS5
                                                                                             socks5
MED   net/ssh                          Uses crypto/ssh to connect to the SSH (secure shell)  crypto/ssh
                                       service
MED   net/stat                         Uses 'netstat' for network information                netstat
MED   net/upload                       uploads files                                         UPLOAD
                                                                                             Upload
                                                                                             upload
MED   net/url/encode                   encodes URL, likely to pass GET variables             urlencode
MED   net/url/request                  requests resources via URL                            http.request
                                                                                             net/url
MED   process/find                     Finds program in process table                        pgrep
MED   process/username/get             returns the user name running this process            whoami
MED   ref/path/etc/hosts               references /etc/hosts                                 /etc/hosts
MED   ref/path/hidden                  hidden path generated dynamically                     %s/.ssh
MED   ref/path/home                    peferences path within /home                          /home/sha2561.32.11.33.01.33.11.33.21.34.01.34.11.3…
MED   ref/path/relative                references and possibly executes relative path        ./jb
                                                                                             ./line
                                                                                             ./pipe
MED   ref/path/root                    path reference within /root                           /root/linuxrc/sbin/hwclock/sbin/ipneigh/sbin/iprout…
MED   ref/path/usr/local               path reference within /usr/local/bin                  /usr/local/bin
MED   ref/words/server_address         references a 'server address', possible C2 client     serverAddress
MED   secrets/keychain                 May access the macOS keychain                         Keychain
                                                                                             keychain
MED   secrets/ssh                      accesses SSH configuration and/or keys                /.ssh/known_hosts
                                                                                             found.ssh
                                                                                             plumbing/object.sshSignatureFormat
                                                                                             repository.ssh
                                                                                             ssh.sshConn
                                                                                             ssh_config.sshLexStateFn
                                                                                             ssh_config.sshLexer
                                                                                             ssh_config.sshParser
                                                                                             …
MED   security_controls/linux/selinux  selinux                                               setenforce
MED   shell/background/sleep           calls sleep and runs shell code in the background     #!
                                                                                             2>&1 &
                                                                                             nohup
MED   shell/exec                       executes shell                                        /bin/bash
                                                                                             /bin/sh
CRIT  third_party/                     Detection patterns for the tool 'RDPassSpray' taken   netcat
      mthcht_thk_yara_rules            from the ThreatHunting-Keywords github project, by
                                       @mthcht
---------------------------------------------------------------------------------------------------------------------------------------------------

Markdown (screenshot of the headers for brevity): CleanShot 2024-05-02 at 19 33 33@2x

egibs commented 2 months ago

I can work on appending the file name to the original path.

Edit: Added in 2b61953 (#180).

egibs commented 2 months ago

Improved the diff output in 0a275a7 (#180). It would be nice to store both original paths, though. 🤔

❯ go run . --diff ~/Downloads/apko_tar_gzs/apko_0.13.2_darwin_amd64.tar.gz ~/Downloads/apko_tar_gzs_2/apko_0.13.2_darwin_arm64.tar.gz
Moved: ../../../../../var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/apko_0.13.2_darwin_amd64.tar.gz2511095900/apko_0.13.2_darwin_amd64/apko -> ../../../../../var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/apko_0.13.2_darwin_arm64.tar.gz178042758/apko_0.13.2_darwin_arm64/apko (score: 0.941791)

Original Path: /Users/egibs/Downloads/apko_tar_gzs_2/apko_0.13.2_darwin_arm64.tar.gz > apko

+++ ADDED: 3 behavior(s) +++

------------------------------------------------------------------------------
RISK  KEY                          DESCRIPTION                      EVIDENCE
------------------------------------------------------------------------------
+LOW  process/chdir                changes working directory        cd H2l
+MED  net/bpf                      BPF (Berkeley Packet Filter)     bpf
+MED  security_controls/linux/ufw  interacts with the ufw firewall  ufw
------------------------------------------------------------------------------

Edit: even better output added in 532088f (#180) (full Markdown rendered as a showcase):

Moved: ../../../../../var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/apko_0.13.2_darwin_amd64.tar.gz2494270430/apko_0.13.2_darwin_amd64/apko -> ../../../../../var/folders/3g/88131l9j11x995ppjbxsvhbh0000gn/T/apko_0.13.2_darwin_arm64.tar.gz2377260856/apko_0.13.2_darwin_arm64/apko (similarity: 0.95)

Original Path (From): /Users/egibs/Downloads/apko_tar_gzs/apko_0.13.2_darwin_amd64.tar.gz > apko

Original Path (To): /Users/egibs/Downloads/apko_tar_gzs_2/apko_0.13.2_darwin_arm64.tar.gz > apko

3 new behaviors

RISK	KEY	DESCRIPTION	EVIDENCE
+MEDIUM	net/bpf	BPF (Berkeley Packet Filter)	bpf
+MEDIUM	security_controls/linux/ufw	interacts with the ufw firewall	ufw
+LOW	process/chdir	changes working directory	cd H2l

egibs commented 2 months ago

I'm going to sit on this and ideate a bit more.

chainguard-dev / bincapz