search

volatilityfoundation / volatility3

Volatility 3.0 development
http://volatilityfoundation.org/
Other
2.61k stars 447 forks source link

How do I allow to download windows symbol table? #986

Closed middleware99 closed 1 year ago

middleware99 commented 1 year ago

By mistake I disallowed to download windows symbol table, and I have no idea how to turn it on again. Don't remember when it was - probably during first volatility usage. I tried to download symbols manually from https://downloads.volatilityfoundation.org/volatility3/symbols/windows.zip and then paste it to volatility3/symbols. It reads symbols, but the error is the same. I attach command and error output:

command:

python .\vol.py -f memory.vmem windows.info

error:

Volatility 3 Framework 2.5.0
Progress:  100.00               PDB scanning finished
Unsatisfied requirement plugins.Info.kernel.layer_name:
Unsatisfied requirement plugins.Info.kernel.symbol_table_name:

A translation layer requirement was not fulfilled.  Please verify that:
        A file was provided to create this layer (by -f, --single-location or by config)
        The file exists and is readable
        The file is a valid memory image and was acquired cleanly

A symbol table requirement was not fulfilled.  Please verify that:
        The associated translation layer requirement was fulfilled
        You have the correct symbol file for the requirement
        The symbol file is under the correct directory or zip file
        The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.Info.kernel.layer_name', 'plugins.Info.kernel.symbol_table_name']

However as I mentioned at the beginning I'm mostly concerned in turning on option to allow to download pdb symbols from Microsoft Store, or wherever there are placed in. I spent an hour on searching in web with no result.

Thank you in advance for any help in that field!

ikelos commented 1 year ago

I don't believe there's currently a way to disable the downloading of symbol tables, other than specifying --offline during a run of volatility3. I know there's a hope at some point to allow saving CLI parameters to configuration file, but at the moment that's not implemented.

You can get more details as to what's going wrong with the specific memory image by using vol.py -vvv rather than vol.py. You'll get a lot of debugging information, which you can append to this issue, or include as a snippet on the volatility slack channel if you want more interactive support.

Volatility won't try to go to the internet if it isn't able to identify the remote operating system (or if it identifies and believes it already has the correct symbol table for it). It might be worth emptying out your symbols directory to make it think that all symbols are missing and try to recreate them.

Please let us know how you get on, so we can give you more help or close this issue off, thanks. 5:)

middleware99 commented 1 year ago

Hi, thanks for quick response!

I attach verbose errors:

command:

python .\vol.py -f memory.vmem -vvv windows.info

output:

Volatility 3 Framework 2.5.0
INFO     volatility3.cli: Volatility plugins path: ['E:\\volatility3\\volatility3\\plugins', 'E:\\volatility3\\volatility3\\framework\\plugins']
INFO     volatility3.cli: Volatility symbols path: ['E:\\volatility3\\volatility3\\symbols', 'E:\\volatility3\\volatility3\\framework\\symbols']
INFO     volatility3.framework.automagic: Detected a windows category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Info
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.windows: Detecting Self-referential pointer for recent windows
DEBUG    volatility3.framework.automagic.windows: Max pointer for hit with test DtbSelfRef64bit not met: 0x23ffffa00 > 0x1ffffffff
DEBUG    volatility3.framework.automagic.windows: Older windows fixed location self-referential pointers
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: TypeError - Layer is not the required Architecture: FileLayer
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['FileLayer']
INFO     volatility3.framework.automagic: Running automagic: WinSwapLayers
INFO     volatility3.framework.automagic: Running automagic: KernelPDBScanner
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
INFO     volatility3.framework.automagic.pdbscan: No suitable kernels found during pdbscan
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder
INFO     volatility3.framework.automagic: Running automagic: KernelModule
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Info.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.kernel.symbol_table_name

Unsatisfied requirement plugins.Info.kernel.layer_name:
Unsatisfied requirement plugins.Info.kernel.symbol_table_name:

A translation layer requirement was not fulfilled.  Please verify that:
        A file was provided to create this layer (by -f, --single-location or by config)
        The file exists and is readable
        The file is a valid memory image and was acquired cleanly

A symbol table requirement was not fulfilled.  Please verify that:
        The associated translation layer requirement was fulfilled
        You have the correct symbol file for the requirement
        The symbol file is under the correct directory or zip file
        The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.Info.kernel.layer_name', 'plugins.Info.kernel.symbol_table_name']

I don't believe there's currently a way to disable the downloading of symbol tables, other than specifying --offline during a run of volatility3.

I didn't mention it, but I refer to Windows popup about allowing or disallowing to download pdb from Microsoft Store. Its not a flag --offline feature, which btw I am aware of existing. I think this Windows popup is triggered during first time attempt to download pdb symbols. By mistake I might denied it and now it won't poupup again. I wonder how to reenable it from system settings or make it appear again.

ikelos commented 1 year ago

Hiya, so volatility builds up to instantiating a windows image. It first tries to locate the directory table base (DTB, the main map for the intel layer), once it has that. There's about 3 different methods find the correct value and volatility will try all three. Once it's found that, it can scan through all of memory looking for the windows kernel and try to identify the specific kernel, so it can download the correct ISF JSON file.

In this instance, it hasn't been able to identify the DTB correctly as seen at:

DEBUG    volatility3.framework.automagic.windows: Detecting Self-referential pointer for recent windows
DEBUG    volatility3.framework.automagic.windows: Max pointer for hit with test DtbSelfRef64bit not met: 0x23ffffa00 > 0x1ffffffff
DEBUG    volatility3.framework.automagic.windows: Older windows fixed location self-referential pointers
...
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['FileLayer']

The error suggests that it found a possible DTB, but that's it's outside the range of physical memory (the size of the image provided) which looks to be an 8Gb image? Without having successfully identified that value, Volatility can't make much progress (and without knowing the DTB, it wasn't able to find a suitable kernel: INFO volatility3.framework.automagic.pdbscan: No suitable kernels found during pdbscan

I'm not sure what else to suggest. If you happened to identify the DTB during acquisition, there's ways of providing it to volatility as a JSON configuration file, but otherwise it's not clear that memory image can be analyzed properly. If you have other memory images, you could check that they work to make sure it isn't your installation (but there's nothing in the output to suggest that).

ikelos commented 1 year ago

I didn't mention it, but I refer to Windows popup about allowing or disallowing to download pdb from Microsoft Store. Its not a flag --offline feature, which btw I am aware of existing. I think this Windows popup is triggered during first time attempt to download pdb symbols. By mistake I might denied it and now it won't poupup again. I wonder how to reenable it from system settings or make it appear again.

Since volatility isn't windows specific, and so can't pop up any notification itself, that's probably not volatility that's doing it. I'd check the windows firewall under the control panel to figure out if there's been a block rule made that you need to remove? Regardless, this specific image is unlikely to have caused volatility to try to access the internet since it could locate a signature to go to the Microsoft servers to find.

middleware99 commented 1 year ago

The error suggests that it found a possible DTB, but that's it's outside the range of physical memory (the size of the image provided) which looks to be an 8Gb image?

it was that size, your're right

I've checked volatility against memory dump from physical station (made using FTK Imager) and it works fine, so I assume that it might be problem with my .vmem files as you suggest somewhere. I think that there's no more need to deal with that issue, so you can close it. Thanks for your attention!

Lastly I want to show a popup message, which I think is what I was talking about. I'm not 100% sure if this is it, I just found it in google graphics.

TZPuRBt

ikelos commented 1 year ago

Just a note, with .vmem files, they can sometimes require an additional file .vmss or .vmsn which contains metadata about how the file is laid out (and could result in a larger "physical" memory image than just the file size. If you can find that, it should have the same name but different extension and be right next to the original file. Unfortunately vmware haven't provided a good way of telling whether a .vmem is just a raw file, or one that requires additional metadata. 5:S

As to the message, volatility doesn't have any windows specific GUI capabilities, so that's being launched by something else? It's possible you got a firewall warning from Windows Firewall which might then block all outbound requests by python or volatility, so as above, check the windows firewall. I'm going to close this off as suggested, but feel free to contact us again, or on slack if you need more support. 5:)