Use different library for libvmi DLL profile generation

psrok1 commented 2 months ago

There are few major problems with current drakpdb-based implementation:

it uses Rekall format that is deprecated and not really supported by recent versions of Drakvuf
drakpdb lacks support for various leaf types (the most important is LF_BITFIELD that is used by recent versions of Drakvuf to identify VAD)
https://github.com/moyix/pdbparse is no longer actively developed and latest stable version has known issues. We're pinned to the specific commit in this repository and that way of version pinning is not supported by PyPi. It means that as long as we're fetching pdbparse from git, we can't publish drakvuf-sandbox package on PyPi.

Drakvuf and libvmi documentation supports Volatility3 IST format and recommends using volatility3.framework.symbols.pdbconv internal module for parsing PDB into profile (https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/symbols/windows/pdbconv.py). The problem is that volatility3 is focused on parsing kernel structures and it doesn't work well for user-mode DLLs.

Here is example of exception that is thrown when we try to parse ole32.pdb from Windows 10:

# python3 -m volatility3.framework.symbols.windows.pdbconv -f ole32.pdb
Traceback (most recent call last):ading TPI layer
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 1073, in <module>
    converted_json = convertor.get_json()
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 505, in get_json
    self.read_necessary_streams()
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 212, in read_necessary_streams
    self.read_pdb_info_stream()
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 447, in read_pdb_info_stream
    self.read_ipi_stream()
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 236, in read_ipi_stream
    type_references = self._read_info_stream(4, "IPI", ipi_list)
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 285, in _read_info_stream
    output, consumed = self.consume_type(module, offset, length)
  File "/opt/venv/lib/python3.9/site-packages/volatility3/framework/symbols/windows/pdbconv.py", line 821, in consume_type
    raise TypeError(f"Unhandled leaf_type: {leaf_type}")
TypeError: Unhandled leaf_type: 5636

It would be OK if volatility3 would ignore unimplemented leaf types (like drakpdb does). Or we can make a PR to the volatility3 that implements missing leaf types. But the problem is still the same: we can't rely on vol3 support for DLLs and we don't have any stable release that magically resolves our issues with parsing DLLs.

That's why we need to think about an alternative way to produce good enough profile in vol3 IST format, without implementing yet another PDB parser :sweat_smile:

psrok1 commented 2 months ago

Actually, even backported version has problems with drakpdb. Debug log from startup for Win10:

1721918603.367763 Windows kernel base address is 0xfffff80049800000
1721918603.367784 Failed to find address for symbol KiInitialPCR
1721918603.367977 Failed to find offset for _EPROCESS:Wow64Process
1721918603.367992 Failed to find offset for VadRoot:BalancedRoot
1721918603.368038 Failed to find offset for _MMVAD:LeftChild
1721918603.368096 Failed to find offset for _MMVAD:RightChild
1721918603.368387 Failed to find offset for _KPCR:PrcbData
1721918603.368442 Failed to find offsets for array of structure names and subsymbols.
1721918603.368449 Failed to find offsets for of bitfield: _MMVAD_FLAGS:Protection.
1721918603.368467 Failed to find offsets for of bitfield: _MMVAD_FLAGS:MemCommit.
1721918603.368472 Failed to find offsets for of bitfield: _MMVAD_FLAGS1:MemCommit.
1721918603.368476 Failed to find offsets for of bitfield: _MMVAD_FLAGS:VadType.
1721918603.368479 Failed to find offsets for of bitfield: (null):(null).
1721918603.368483 Failed to find offsets for of bitfield: _MMVAD_FLAGS:CommitCharge.
1721918603.368497 Failed to find offsets for of bitfield: _MMVAD_FLAGS1:CommitCharge.

CERT-Polska / drakvuf-sandbox

Use different library for libvmi DLL profile generation #937