search

volatilityfoundation / volatility3

Volatility 3.0 development
http://volatilityfoundation.org/
Other
2.72k stars 462 forks source link

"Symbol table requirement not filled"? #843

Closed d-millar closed 1 year ago

d-millar commented 2 years ago

Describe the bug Presumably user error, but none of the Linux plugins are working for me, i.e. they run but produce no results.

Context Volatility Version: Volatility 3 Framework 2.3.1 (cloned from github, current) Operating System: running Volatility from Ubuntu 20.04.5 against an Ubuntu 22.04.1 target Python Version: 3.8.10 Suspected Operating System: not sure what this is asking (target dump is Ubuntu 22.04.1) Command: python3 vol.py -f ubuntu.dmp linux.pslist.PsList (or really anything)

To Reproduce Steps to reproduce the behavior: (1) virt-manager with Ubuntu 22.04 client (2) virsh dump ubuntu22.04 ubuntu.dmp (3) python vol.py -f ubuntu.dmp banners.Banners (4) download linux-image-unsigned-5.15.0-48-generic-dbgsym_5.15.0-48.54_amd64.ddeb from http://ddebs.ubuntu.com/pool/main/l/linux/ (5) dpkg -x linux-image-unsigned-5.15.0-48-generic-dbgsym_5.15.0-48.54_amd64.ddeb (6) /x/git/dwarf2json/dwarf2json linux --elf ./usr/lib/debug/boot/vmlinux-5.15.0-48-generic > linux-image-5.15.0-48-generic-amd64.json (7) cp linux-image-5.15.0-48-generic-amd64.json /x/git/volatility3/volatility3/symbols/linux (8) do the same for 5-15.0-46 (banners returned both)

Expected behavior Some content below the following 3 lines: Volatility 3 Framework 2.3.1 Progress: 100.00 Stacking attempts finished
OFFSET (V) PID TID PPID COMM

Additional information Banners result: 0x248633e0 Linux version 5.15.0-48-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 (Ubuntu 5.15.0-48.54-generic 5.15.53) 0x25a9ab76 Linux version 5.15.0-48-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 (Ubuntu 5.15.0-48.54-generic 5.15.53)3) 0x28e21250 Linux version 5.15.0-46-generic (buildd@lcy02-amd64-115) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 (Ubuntu 5.15.0-46.49-generic 5.15.39) 0x28e734f2 Linux version 5.15.0-46-generic (buildd@lcy02-amd64-115) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 (Ubuntu 5.15.0-46.49-generic 5.15.39)

IsfInfo result: file:///x/git/volatility3/volatility3/symbols/windows/ntkrnlmp.pdb/88B8A4B5CFBA32B6F31E476E95096C28-1.json.xz True (cached) 14 0 35039 247 b'ntkrnlmp.pdb|88B8A4B5CFBA32B6F31E476E95096C28|1' file:///x/git/volatility3/volatility3/symbols/linux/linux-image-5.15.0-46-generic-amd64.json True (cached) 19 0 194117 2076 b'Linux version 5.15.0-46-generic (buildd@lcy02-amd64-115) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 (Ubuntu 5.15.0-46.49-generic 5.15.39)\n\x00' file:///x/git/volatility3/volatility3/symbols/linux/linux-image-5.15.0-48-generic-amd64.json True (cached) 19 0 194007 2084 b'Linux version 5.15.0-48-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 (Ubuntu 5.15.0-48.54-generic 5.15.53)\n\x00'

PsList w/ -vvv: Volatility 3 Framework 2.3.1 INFO volatility3.cli: Volatility plugins path: ['/x/git/volatility3/volatility3/plugins', '/x/git/volatility3/volatility3/framework/plugins'] INFO volatility3.cli: Volatility symbols path: ['/x/git/volatility3/volatility3/symbols', '/x/git/volatility3/volatility3/framework/symbols'] INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available DEBUG volatility3.framework: No module named 'yara' DEBUG volatility3.framework: Failed to import module volatility3.plugins.yarascan based on file: /x/git/volatility3/volatility3/framework/plugins/yarascan.py INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available DEBUG volatility3.framework: No module named 'yara' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.svcscan based on file: /x/git/volatility3/volatility3/framework/plugins/windows/svcscan.py INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available DEBUG volatility3.framework: No module named 'yara' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.vadyarascan based on file: /x/git/volatility3/volatility3/framework/plugins/windows/vadyarascan.py INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available DEBUG volatility3.framework: No module named 'yara' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.mftscan based on file: /x/git/volatility3/volatility3/framework/plugins/windows/mftscan.py INFO volatility3.cli: The following plugins could not be loaded (use -vv to see why): volatility3.plugins.windows.mftscan, volatility3.plugins.windows.svcscan, volatility3.plugins.windows.vadyarascan, volatility3.plugins.yarascan INFO volatility3.framework.automagic: Detected a linux category plugin INFO volatility3.framework.automagic: Running automagic: ConstructionMagic Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList INFO volatility3.framework.automagic: Running automagic: SymbolCacheMagic INFO volatility3.framework.automagic: Running automagic: LayerStacker Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name DEBUG volatility3.framework.layers.qemu: QEVM header found DEBUG volatility3.framework.layers.qemu: QEVM header found DEBUG volatility3.framework.automagic.linux: Identified banner: b'Linux version 5.15.0-48-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 (Ubuntu 5.15.0-48.54-generic 5.15.53)\n\x00' DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!netns_ipvs DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_pkg_stats DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_rcv_lists_stats DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_dev_rcv_lists DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_route DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!sctp_mib DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_stats_rsn DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_stats DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!dn_dev DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!garp_port DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!macsec_ops DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!mctp_dev DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_dev DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!mrp_port DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!tipc_bearer DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_conn DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cached_keys DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cqm_config DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_internal_bss DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!libipw_device DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo DEBUG volatility3.framework.symbols: Unresolved reference: LintelStacker1!dsa_8021q_context DEBUG volatility3.framework.automagic.linux: Scanners could not determine any ASLR shifts, using 0 for both DEBUG volatility3.framework.automagic.linux: DTB was found at: 0x2e10000 Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.PsList.kernel.layer_name.memory_layer Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList.kernel Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.PsList DEBUG volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'FileLayer'] INFO volatility3.framework.automagic: Running automagic: SymbolFinder
INFO volatility3.framework.automagic: Running automagic: LinuxSymbolFinder Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.PsList.kernel.symbol_table_name DEBUG volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 5.15.0-48-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 (Ubuntu 5.15.0-48.54-generic 5.15.53)\n\x00' DEBUG volatility3.framework.automagic.symbol_finder: Using symbol library: file:///x/git/volatility3/volatility3/symbols/linux/linux-image-5.15.0-48-generic-amd64.json INFO volatility3.framework.automagic: Running automagic: KernelModule

OFFSET (V) PID TID PPID COMM DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!netns_ipvs DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_pkg_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_rcv_lists_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_dev_rcv_lists DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_route DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sctp_mib DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_stats_rsn DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!macsec_ops DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mctp_dev DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_dev DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mrp_port DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tipc_bearer DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cqm_config DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dsa_8021q_context

ikelos commented 2 years ago

Hi there, it looks like volatility is successfully identifying the image and kernel, so the next best guess is that the ASLR determination hasn't worked out properly for some reason. @atcuno any thoughts?

3hhh commented 1 year ago

I have very similar results for a debian-11 memory dump generated via virsh -c xen:// dump vm vm.dump --live on a Xen host:

On a debian-11 Xen VM with the same kernel as the dumped VM I ran the following:

> sudo apt install linux-image-amd64-dbg
>./dwarf2json linux --elf /usr/lib/debug/boot/vmlinux-[kernel version]-amd64 --system-map /usr/lib/debug/boot/System.map-[kernel version]-amd64 > [volatility path]/volatility3/symbols/out.json
> python3 vol.py isfinfo
Volatility 3 Framework 2.4.0
Progress:  100.00               PDB scanning finished  
URI     Valid   Number of base_types    Number of types Number of symbols       Number of enums Identifying information

file:///home/user/volatility3-2.4.0/volatility3/symbols/out.json        Unknown 18      0       143295  1529    b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'

> python3 vol.py -vvv -f vm.dump linux.lsmod
Volatility 3 Framework 2.4.0
INFO     volatility3.cli: Volatility plugins path: ['/home/user/volatility3-2.4.0/volatility3/plugins', '/home/user/volatility3-2.4.0/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/user/volatility3-2.4.0/volatility3/symbols', '/home/user/volatility3-2.4.0/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a linux category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo
DEBUG    volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical 65e4000 virtual 1ac00000
DEBUG    volatility3.framework.automagic.linux: DTB was found at: 0x8bee000
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name.memory_layer
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: file:///home/user/volatility3-2.4.0/volatility3/symbols/out.json
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
INFO     volatility3.framework.automagic: Running automagic: KernelModule

Offset  Name    Size
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo

So no output as it complains about missing symbols even though isfinfo shows there are some in the json file. Grepping through out.json also shows entries for assoc_array_ptr and the others.

Are any issues known with Xen memory dumps?

I can provide a copy of the dump to a maintainer, if needed (there's no relevant data in it).

ikelos commented 1 year ago

The unresolved symbols are not errors, they're debugging messages. It's more likely an issue in the means to cross connect all the structures together in python as a symbol table object. I wouldn't get hung up on those, you'd more likely see a specific error after those if volatility tried to lookup a symbol that didn't exist.

I don't know of any issues specifically with Xen dumps, but I don't know how Xen dumps memory and whether it does anything to the data that might throw off relative offsets and so on. Since I believe they both use qemu/KVM at their heart, I don't think there should be anything in particular wrong with them. My best guess is still the ALSR not correctly set and therefore throwing off the locations of various structures.

I don't know how important it is but:

/x/git/dwarf2json/dwarf2json linux --elf ./usr/lib/debug/boot/vmlinux-5.15.0-48-generic > linux-image-5.15.0-48-generic-amd64.json

Might not be getting symbol locations but just the structures. It's important to include the System.map file as well. I'd expect volatility to say it couldn't find the symbol it was looking for if it weren't there, but that's another possibility...

eve-mem commented 1 year ago

@d-millar - with virsh can you try with the --memory-only option. So it would be virsh dump --memory-only ubuntu22.04 ubuntu.dmp --format elf. You shouldn't need to specify the format, but at least this way you can be sure.

Edit - This may also help you @3hhh

ikelos commented 1 year ago

Ok, thanks to some awesome sleuthing and help by @eve-mem I think this is now fixed as of commit cd89e39. It turned out to be a problem with our QEVM parser, which meant the files were being treated like raw image files (because the last byte of the configuration was chopped off, making it invalid JSON). Please could you test it and let me know so we can close out this issue...

3hhh commented 1 year ago

Hm I checked out the develop branch and tested with that, but unfortunately it didn't fix the issue yet:

> python3 vol.py -vvv -f vm.dump linux.lsmod
Volatility 3 Framework 2.4.1
INFO     volatility3.cli: Volatility plugins path: ['/home/user/volatility/volatility3/plugins', '/home/user/volatility/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/user/volatility/volatility3/symbols', '/home/user/volatility/volatility3/framework/symbols']
INFO     volatility3.framework.automagic: Detected a linux category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo
DEBUG    volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical 65e4000 virtual 1ac00000
DEBUG    volatility3.framework.automagic.linux: DTB was found at: 0x8bee000
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name.memory_layer
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: file:///home/user/volatility/volatility3/symbols/out.json
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
INFO     volatility3.framework.automagic: Running automagic: KernelModule

Offset  Name    Size
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo

It's a Qubes OS Xen pvh domain with qemu stubdom (i.e. qemu runs inside the VM).

If you have some contact address, I can upload the image somewhere and send the decryption password to the contact address.

3hhh commented 1 year ago

Side note: virsh -c xen:// dump vm vm.dump --memory-only --format elf throws error: unsupported flags (0x10) in function libxlDomainCoreDump for me.

> file vm.dump 
vm.dump: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), no program header
eve-mem commented 1 year ago

Does that elf memory dump parse in vol3?

If you run your first dump (e.g. not the elf one) with all 7 vs do you get any errors? E.g. -vvvvvvv any python crashes?

Might be a big output so you shouldn't need to post it all here.

3hhh commented 1 year ago

file vm.dump vm.dump: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), no program header

This is the first dump from virsh -c xen:// dump vm vm.dump --live. The other call just generates the error and no dump at all.

Anyway, -vvvvvvv output looks like it may have some issue:

INFO     volatility3.cli: Volatility plugins path: ['/home/user/volatility/volatility3/plugins', '/home/user/volatility/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/home/user/volatility/volatility3/symbols', '/home/user/volatility/volatility3/framework/symbols']
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/plugins, /home/user/volatility/volatility3/framework/plugins
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/automagic
Level 7  volatility3.cli: Cache directory used: /home/user/.cache/volatility3
INFO     volatility3.framework.automagic: Detected a linux category plugin
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
Level 6  volatility3.framework.symbols.intermed: Searching for symbols in /home/user/volatility/volatility3/symbols, /home/user/volatility/volatility3/framework/symbols
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 7  volatility3.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, VolatilityHandler, JarHandler, OfflineHandler, LeechCoreHandler
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 6  volatility3.framework.symbols.intermed: Searching for symbols in /home/user/volatility/volatility3/symbols, /home/user/volatility/volatility3/framework/symbols
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
Level 7  volatility3.framework.automagic.stacker: Exception during stacking: No ELF segments defined in FileLayer
Level 6  volatility3.framework.automagic.stacker: Traceback (most recent call last):

  File "/home/user/volatility/volatility3/framework/automagic/stacker.py", line 213, in stack_layer
    new_layer = stacker.stack(context, initial_layer, progress_callback)

  File "/home/user/volatility/volatility3/framework/layers/elf.py", line 122, in stack
    return Elf64Layer(context, new_name, new_name)

  File "/home/user/volatility/volatility3/framework/layers/elf.py", line 34, in __init__
    super().__init__(context, config_path, name)

  File "/home/user/volatility/volatility3/framework/layers/segmented.py", line 38, in __init__
    self._load_segments()

  File "/home/user/volatility/volatility3/framework/layers/elf.py", line 69, in _load_segments
    raise ElfFormatException(

volatility3.framework.layers.elf.ElfFormatException: No ELF segments defined in FileLayer

Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo
DEBUG    volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical 65e4000 virtual 1ac00000
DEBUG    volatility3.framework.automagic.linux: DTB was found at: 0x8bee000
Level 8  volatility3.framework.automagic.stacker: Stacked IntelLayer using LinuxIntelStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 6  volatility3.framework.layers.elf: Exception: Offset 0x0 does not exist within the base layer
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Lsmod.kernel.layer_name.memory_layer
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 6  volatility3.framework: Importing from the following paths: /home/user/volatility/volatility3/framework/layers
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Lsmod
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Lsmod.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 5.10.0-20-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2 (2022-12-13)\n\x00'
DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: file:///home/user/volatility/volatility3/symbols/out.json
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
INFO     volatility3.framework.automagic: Running automagic: KernelModule
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!netns_ipvs
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!macsec_ops
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mrp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tipc_bearer
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_pkg_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_rcv_lists_stats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_dev_rcv_lists
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_route
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sctp_mib
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cqm_config
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phy_led_trigger
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo
Volatility 3 Framework 2.4.1

Offset  Name    Size
eve-mem commented 1 year ago

Ah I understand, its different to what I thought the problem was.

That error also looks like a different issue to what @ikelos fixed yesterday. Looks like a possible problem with the elf layer.

eve-mem commented 1 year ago

Essentially vol3 isn't able to understand that elf format for some reason. Which means it's being treated as a flat file, but that means the offsets are wrong which is why you get no output. Likely it finds the init task, follows the pointer to where the first process is, but it's in the wrong place and so it stops.

This is the main entry in the log that shows the issue:

Stacked layers: ['IntelLayer', 'FileLayer']

With your elf file you'd want to see that layer too.

3hhh commented 1 year ago

On 1/7/23 19:17, Eve wrote:

Essentially vol3 isn't able to understand that elf format for some reason. Which means it's being treated as a flat file, but that means the offsets are wrong which is why you get no output. Likely it finds the init task, follows the pointer to where the first process is, but it's in the wrong place and so it stops.

This is the main entry in the log that shows the issue:

Stacked layers: ['IntelLayer', 'FileLayer']

With your elf file you'd want to see that layer too.

I see.

Is vol3 supposed to support the format?

And if so, is there anything left that I need to provide to help you guys fix the issue?

eve-mem commented 1 year ago

Yes elfs should be supported (I use them all the time) - I think at this point its best to get that mem dump in the hands of someone who understands the elf layer (not me I'm afraid..!)

ikelos commented 1 year ago

We parse the ELF headers ourselves, so it's possible we do it wrong. We're saying we can't enumerate any segments in the file (No ELF segments defined in FileLayer). So it might be interesting to see what something like readelf -a says about the file. My concern is that Xen threw an error during the saving. I'm also starting to get confused following which file formats we believe which files are written in. I think the timeline is:

@d-millar Had the original issue, his run found a QEVM header, and that may match the thing we fixed @3hhh thought they had a similar issue, but that didn't say QEVM header found, and may be totally different No additional dumps could be generated because --format elf didn't work, so we still don't know whether --memory-only will help, and we still don't know why it seems to be writing ELF files, even without the --format elf flag...

Hopefully that's an accurate summary of what's gone on? @3hhh please could I ask you to file a separate issue to we can keep track of the two problems separately? In which case, this is still awaiting verification from @d-millar on the original issue...

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 200 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 60 days since being marked as stale.