volatilityfoundation / volatility3

Volatility 3.0 development
http://volatilityfoundation.org/
Other
2.61k stars 447 forks source link

Unresolved reference: symbol_table_name for Ubuntu 22.04 symbols #963

Closed Rufmord closed 1 year ago

Rufmord commented 1 year ago

Unresolved reference: symbol_table_name for Ubuntu 22.04 symbols I have a vmem file from Ubuntu 22.04 and the generated symbols table but I get an unresolved reference in any output.

Context Volatility Version: 2.4.2 Operating System: kali-linux-2023.1 image for VMware Python Version: 3.11.2 Suspected Operating System: Ubuntu 22.04 Command: linux.* (any, e. g. bash, pslist, ...)

To Reproduce Steps to reproduce the behavior:

  1. Using the command python3 vol.py -vvvvvvv -f ../Forensics-Symbols-8a2b7a15.vmem linux.bash I get the following output
    
    Volatility 3 Framework 2.4.2
    INFO     volatility3.cli: Volatility plugins path: ['/home/kali/volatility3/volatility3/plugins', '/home/kali/volatility3/volatility3/framework/plugins']
    INFO     volatility3.cli: Volatility symbols path: ['/home/kali/volatility3/volatility3/symbols', '/home/kali/volatility3/volatility3/framework/symbols']
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/plugins, /home/kali/volatility3/volatility3/framework/plugins
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/automagic
    Level 7  volatility3.cli: Cache directory used: /home/kali/.cache/volatility3
    INFO     volatility3.framework.automagic: Detected a linux category plugin
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework.automagic.construct_layers: Construction Exception occurred: Unexpected config value found: None
    INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
    Level 6  volatility3.framework.symbols.intermed: Searching for symbols in /home/kali/volatility3/volatility3/symbols, /home/kali/volatility3/volatility3/framework/symbols
    INFO     volatility3.framework.automagic: Running automagic: LayerStacker
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 7  volatility3.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, VolatilityHandler, JarHandler, OfflineHandler
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
    Level 6  volatility3.framework.layers.elf: Exception: Bad magic 0xf000ff53 at file offset 0x0
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using XenCoreDumpStacker
    Level 6  volatility3.framework.layers.xen: Exception: Bad magic 0xf000ff53 at file offset 0x0
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
    Level 6  volatility3.framework.layers.vmware: Metadata found: VMSS (False) or VMSN (False)
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
    DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 5.19.0-42-generic (buildd@lcy02-amd64-074) (x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 (Ubuntu 5.19.0-42.43~22.04.1-generic 5.19.17)\n\x00'
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!netns_ipvs
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_pkg_stats
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_rcv_lists_stats
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!can_dev_rcv_lists
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_route
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sctp_mib
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_stats_rsn
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_stats
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!dn_dev
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!garp_port
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!macsec_ops
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mpls_dev
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mrp_port
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!tipc_bearer
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_conn
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cached_keys
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_cqm_config
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!libipw_device
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!dsa_8021q_context
    DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!cfg80211_internal_bss
    DEBUG    volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical 140800000 virtual 31000000
    DEBUG    volatility3.framework.automagic.linux: DTB was found at: 0x143810000
    Level 8  volatility3.framework.automagic.stacker: Stacked IntelLayer using LinuxIntelStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
    Level 6  volatility3.framework.layers.elf: Exception: Offset 0x0 does not exist within the base layer
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using XenCoreDumpStacker
    Level 6  volatility3.framework.layers.xen: Exception: Offset 0x0 does not exist within the base layer
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
    Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name.memory_layer
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    Level 6  volatility3.framework: Importing from the following paths: /home/kali/volatility3/volatility3/framework/layers
    DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'FileLayer']
    INFO     volatility3.framework.automagic: Running automagic: SymbolFinder  
    INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
    Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
    DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 5.19.0-42-generic (buildd@lcy02-amd64-074) (x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 (Ubuntu 5.19.0-42.43~22.04.1-generic 5.19.17)\n\x00'
    DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: file:///home/kali/volatility3/volatility3/symbols/linux/ubuntu.json
    INFO     volatility3.framework.automagic: Running automagic: KernelModule

PID Process CommandTime Command Level 6 volatility3.framework.symbols.intermed: Searching for symbols in /home/kali/volatility3/volatility3/symbols, /home/kali/volatility3/volatility3/framework/symbols DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!netns_ipvs DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_pkg_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_rcv_lists_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!can_dev_rcv_lists DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_route DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sctp_mib DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_stats_rsn DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_stats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!macsec_ops DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mpls_dev DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mrp_port DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tipc_bearer DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cqm_config DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dsa_8021q_context DEBUG volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss


**Expected behavior**
It should print the bash history or the process list.

**Additional information**
The symbol table was generated on the same VM the vmem file is from. 
I installed `dwarf2json`
```shell
git clone https://github.com/volatilityfoundation/dwarf2json
cd dwarf2json
go build

Added the ddebs file with content

sudo vim /etc/apt/sources.list.d/ddebs.list
deb http://ddebs.ubuntu.com jammy main restricted universe multiverse
deb http://ddebs.ubuntu.com jammy-updates main restricted universe multiverse
deb http://ddebs.ubuntu.com jammy-proposed main restricted universe multiverse

And generated the symbols


wget -O - http://ddebs.ubuntu.com/dbgsym-release-key.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install linux-image-`uname -r`-dbgsym -y
sudo ./dwarf2json linux --elf /usr/lib/debug/boot/vmlinux-5.19.0-42-generic > ~/ubuntu.json
eve-mem commented 1 year ago

It looks like volatility3 couldn't find the metadata files for your memory image. Its this line here

Metadata found: VMSS (False) or VMSN (False)

If you have a vmss or vmsn file and place them along side the image it may 'just work'.

Rufmord commented 1 year ago

Thank you, that helped and I get an output. But I tried the same thing about a month ago without the vmss file and it worked before? What was changed?

ikelos commented 1 year ago

It's still not clear what causes VMware to split the metadata out into a separate file (the VMSS/VMSN file). It's also possible you got lucky and that the data was contiguous and the information from the metadata file didn't have an effect on what you were looking for. If it was exactly the same file, then I agree that's strange, because I don't believe we've done any changes recently that would impact that?

Either way, very glad to hear you got it resolved, I'll mark this as closed but feel free to reopen if you feel it hasn't been resolved. 5:)

Rufmord commented 1 year ago

This is strange, but it works now. Thank you very much. It would also be helpful to write this infmoration in the docs for future usage.