volatilityfoundation / volatility3

Volatility 3.0 development
http://volatilityfoundation.org/
Other
2.61k stars 447 forks source link

Symbol table requirement was not fulfilled. #760

Closed bellohai closed 12 months ago

bellohai commented 2 years ago

Hello guys,

I am new to MacOS RAM analysis. I am facing issue related to symbol table requirement was not fulfilled. I already using dwarf2json to create a new symbol table file for my macbook Monterey and transfer it to the file path: "volatility3/symbols/mac" with the name "kernel_debug_kit_12.0.1_build_21A559.dmg.json.xz" .

image

Below are how i create symbols file and import to Linux Machine: Create symbol

In Mac OS Environment 1) open About this mac to check the mac version and build number 2) Go to https://developer.apple.com/download/all/?q=debug 3) Search for the kernel debug tool that fit mac version and build number 4) install the pkg files into mac 5) Download dwarf2json tool by running command git clone https://github.com/volatilityfoundation/dwarf2json 6) Run command "go build" 7) Run command "dwarf2json mac --macho /library/developer/kdks/KDK_12.0.1_21A559.kdk/system/library/Kernels/kernel.dSYM/Contents/Resources/DWARF/Kernel --macho-symbols /library/developer/kdks/KDK_12.0.1_21A559.kdk/system/library/Kernels/kernel" > Kernel_Debug_Kit_12.0.1_build_21A559.dmg.json 8) use command strings [/path/to/memorydumpfile] | grep -i "Darwin Kernel Version" 9) Copy the output (For my case, Darwin Kernel Version 21.1.0................RELEASE_X86_64) 10) run command "base64 <<< "output"" and copy the output (list of string) 11) open the json generated in step 7, search for "constant_data" and replace the value from the output in step 10 12) run command xz [Kernel_Debug_Kit_12.0.1_build_21A559.dmg.json] 13) save the xz file into external drive

Import Symbol

In Linux OS (volatility machine) 1) Plug in the external drive 2) copy the xz file into the directory volatility3/volatility3/symbols/mac

Question: Did I create the symbols file wrongly ? How can I solve this issue on MacOS Monterey RAM analysis?

ikelos commented 2 years ago

Hiya, so the steps you followed look right, but it's unclear why you then messed with the banner constant_data. Dwarf2json should fill this in automatically unless something went wrong?

If you have the KDK dmg file, then the following script should do everything for you? It extracts the .dmg and locates the appropriate debug kernel.

You can verify that volatility finds the symbol file and how many symbols it contains using the isfinfo plugin. You should expect to see numbers in the columns around these sizes: 18 4826 42765 160, if you don't it suggests the wrong file was used in some way. If the file isn't listed, it suggests it wasn't present in the right location. If it has no banner then something went wrong and dwarf2json didn't extract the banner correctly (and you can try your steps 8 to 12, again, although it's not recommended). Finally, you can use the banners plugin to check that the banner would match the one found in the image...

Hope that helps? If not, please provide a few more diagnostics (the output from isfinfo and banners) to try and figure out what's going on... 5:)

bellohai commented 2 years ago

Hi @ikelos , thank you for your reply.

Below screenshot is the result of "Python3 vol.py isfinfo --filter mac" :

image

Below screenshot is the output of banners:

image

After adding the "Kernel_Debug_Kit_12.0.1_build_21A559.dmg.json.xz" file into file path "volatility3/volatility3/symbols/mac", the output that i get is:

image

Question: How can I solve the issue ?

ikelos commented 2 years ago

Hmmm, that does look as though the banner should be found, although the banner is also mentioned more times than I'd expect, so it's possible it isn't identifying the right one to locate all the other things it needs? You can see more about what volatility is doing (and whether it's matching the banner) by running the same command but with vol.py -vvvvv rather than vol.py. That should tell us whether it's finding the banner, and if so what values it's using for the kernel and so on...

bellohai commented 2 years ago

Hello, I tried to regenerate everything including the symbol file and the memory file. I use the command vol.py -vvvvv.

Below is the issue that i face "zsh: killed " :

┌──(root㉿kali)-[/home/kali/Documents/MacOS] └─# python3 volatility3/vol.py -vvvvv -f /media/sf_D_DRIVE/macramnew.dump mac.bash.Bash

Volatility 3 Framework 2.2.0 INFO volatility3.cli: Volatility plugins path: ['/home/kali/Documents/MacOS/volatility3/volatility3/plugins', '/home/kali/Documents/MacOS/volatility3/volatility3/framework/plugins'] INFO volatility3.cli: Volatility symbols path: ['/home/kali/Documents/MacOS/volatility3/volatility3/symbols', '/home/kali/Documents/MacOS/volatility3/volatility3/framework/symbols'] DEBUG volatility3.framework: No module named 'Crypto' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.hashdump based on file: /home/kali/Documents/MacOS/volatility3/volatility3/framework/plugins/windows/hashdump.py DEBUG volatility3.framework: No module named 'Crypto' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.cachedump based on file: /home/kali/Documents/MacOS/volatility3/volatility3/framework/plugins/windows/cachedump.py DEBUG volatility3.framework: No module named 'Crypto' DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.lsadump based on file: /home/kali/Documents/MacOS/volatility3/volatility3/framework/plugins/windows/lsadump.py INFO volatility3.cli: The following plugins could not be loaded (use -vv to see why): volatility3.plugins.windows.cachedump, volatility3.plugins.windows.hashdump, volatility3.plugins.windows.lsadump Level 7 volatility3.cli: Cache directory used: /root/.cache/volatility3 INFO volatility3.framework.automagic: Detected a mac category plugin INFO volatility3.framework.automagic: Running automagic: ConstructionMagic Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash INFO volatility3.framework.automagic: Running automagic: SymbolBannerCache INFO volatility3.framework.automagic: Running automagic: MacBannerCache INFO volatility3.framework.automagic: Running automagic: LayerStacker Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name Level 7 volatility3.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, VolatilityHandler, JarHandler, OfflineHandler Level 8 volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker Level 8 volatility3.framework.automagic.stacker: Attempting to stack using MacIntelStacker DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64\x00' Level 7 volatility3.framework.automagic.mac: Invalid kalsr_shift found at offset: 151921582 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64\x00' Level 7 volatility3.framework.automagic.mac: Invalid kalsr_shift found at offset: 441504369 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64\x00' Level 7 volatility3.framework.automagic.mac: Invalid kalsr_shift found at offset: 441504468 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64\x00' Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 447238448 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 503134842 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 552926166 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 708952212 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 889294946 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 913227138 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 1040406577 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 1040406676 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 1088853686 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 1418640194 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 1636450786 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 2037139902 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 2718787588 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 2836088072 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE Level 7 volatility3.framework.automagic.mac: Invalid kalsrshift found at offset: 3097759038 DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 PDT 2021; root:xnu-8019.41.5~1/RELEASE zsh: killed python3 volatility3/vol.py -vvvvv -f /media/sf_D_DRIVE/macramnew.dump

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 200 days with no activity.

github-actions[bot] commented 12 months ago

This issue was closed because it has been inactive for 60 days since being marked as stale.