KeyError:3 while running capa on VMRay analysis archive

ForensicITGuy commented 1 month ago

Description

While running capa 7.3.0 on a VMRay analysis archive, capa crashed and provided a stack trace for troubleshooting.

Steps to Reproduce

Obtain VMRay sample archive for sample 2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4 (attached)
Run ./capa -d -f vmray 2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4_analysis_archive.zip
Observe output

Expected behavior:

I expected a capability report.

Actual behavior:

The capa process crashed with stack trace:

DEBUG:capa:--------------------------------------------------------------------------------
DEBUG:capa: Using default embedded rules.
DEBUG:capa: To provide your own rules, use the form:
DEBUG:capa:
DEBUG:capa:     `capa.exe -r ./path/to/rules/  /path/to/mal.exe`.
DEBUG:capa:
DEBUG:capa: You can see the current default rule set here:
DEBUG:capa:
DEBUG:capa:     https://github.com/mandiant/capa-rules
DEBUG:capa:--------------------------------------------------------------------------------
DEBUG:capa.rules:reading rules from directory /tmp/_MEIoxSNmT/rules
DEBUG:capa.rules.cache:loading rule set from cache: /tmp/_MEIoxSNmT/cache/capa-df400217.cache
DEBUG:capa:successfully loaded 901 rules
DEBUG:capa.features.extractors.vmray:file_type: Windows Exe (x86-32), file_path: internal/static_analyses/2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4/objects/files/2f8a7
9b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4
Traceback (most recent call last):
  File "main.py", line 1094, in <module>
  File "main.py", line 956, in main
  File "main.py", line 708, in get_file_extractors_from_cli
  File "loader.py", line 390, in get_file_extractors
  File "features/extractors/vmray/extractor.py", line 122, in from_zipfile
  File "features/extractors/vmray/__init__.py", line 87, in __init__
  File "features/extractors/vmray/__init__.py", line 141, in _compute_process_threads
  File "features/extractors/vmray/__init__.py", line 161, in get_process_os_pid
KeyError: 3
[PYI-1584:ERROR] Failed to execute script 'main' due to unhandled exception!

Versions

capa 7.3.0 5.15.0-1047-aws 20.04.1-Ubuntu x86_64

Additional Information

See attached analysis archive for troubleshooting 2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4_analysis_archive.zip

mr-tz commented 1 month ago

Thanks for the report and archive for testing. We assume all processes are tracked in summary_v2.json but here they are not.

❯ grep -Eo "process_id=\"[0-9]+\"" 2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4_analysis_archive/logs/flog.xml | sort | uniq
process_id="1"
process_id="2"
process_id="3"
process_id="4"
process_id="5"
process_id="6"
process_id="7"
process_id="8"
process_id="9"

❯ grep -Eo "\"monitor_id\": [0-9]+" 2f8a79b12a7a989ac7e5f6ec65050036588a92e65aeb6841e08dc228ff0e21b4_analysis_archive/logs/summary_v2.json | sort | uniq
"monitor_id": 1
"monitor_id": 2
"monitor_id": 4
"monitor_id": 8

flog.xml contains all data, so we may have to just rely on that, e.g.:

<monitor_process ts="109816" process_id="3" image_name="wmiprvse.exe" filename="c:\\windows\\system32\\wbem\\wmiprvse.exe" page_root="0x4a684000" os_pid="0x694" os_integrity_level="0x4000" os_privileges="0x1e60b1e890" monitor_reason="rpc_server" parent_id="2" os_parent_pid="0x200" cmd_line="C:\\Windows\\system32\\wbem\\wmiprvse.exe -Embedding" cur_dir="C:\\Windows\\system32\\" ..."/>

mr-tz commented 1 month ago

now getting:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Capability                                       ┃ Namespace                                       ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ check for time delay via QueryPerformanceCounter │ anti-analysis/anti-debugging/debugger-detection │
│ reference anti-VM strings                        │ anti-analysis/anti-vm/vm-detection              │
│ reference anti-VM strings targeting VirtualBox   │ anti-analysis/anti-vm/vm-detection              │
│ initialize Winsock library (2 matches)           │ communication/socket                            │
│ reference Base64 string                          │ data-manipulation/encoding/base64               │
│ generate random numbers via RtlGenRandom         │ data-manipulation/prng                          │
│ generate random numbers via WinAPI               │ data-manipulation/prng                          │
│ contain a thread local storage (.tls) section    │ executable/pe/section/tls                       │
│ query environment variable (18 matches)          │ host-interaction/environment-variable           │
│ read file on Windows (2 matches)                 │ host-interaction/file-system/read               │
│ write file on Windows                            │ host-interaction/file-system/write              │
│ get system information on Windows (2 matches)    │ host-interaction/os/info                        │
│ modify access privileges                         │ host-interaction/process/modify                 │
│ linked against Go registry library               │ host-interaction/registry                       │
│ resume thread (4 matches)                        │ host-interaction/thread/resume                  │
│ manipulate user privileges                       │ host-interaction/user                           │
│ reference cryptocurrency strings                 │ impact/cryptocurrency                           │
│ link many functions at runtime (3 matches)       │ linking/runtime-linking                         │
│ linked against OpenSSL                           │ linking/static/openssl                          │
│ linked against wolfSSL                           │ linking/static/wolfssl                          │
│ spawn thread to RWX shellcode (2 matches)        │ load-code/shellcode                             │
└──────────────────────────────────────────────────┴─────────────────────────────────────────────────┘

mandiant / capa