volatilityfoundation / volatility3

Volatility 3.0 development
http://volatilityfoundation.org/
Other
2.61k stars 447 forks source link

Write a cross platform plugin ? #868

Open ShellCode33 opened 1 year ago

ShellCode33 commented 1 year ago

Hey, I hope this is the right place to ask such question, sorry if it's not.

I'd like to write a plugin to analyze a specific application on any platform (Windows, Linux and MacOS). The idea would be to have a platform specific entry point that will gather all the needed information, and then dispatch it to a common processing function that will do the same thing regardless of the underlying OS.

I have not been able to find any online material on how to do such thing, feel free to redirect me to any external website.

Note that I'm not familiar at all with volatility's code, this is a first for me.

So, my plugin will depend on the PsList plugin (and probably others later), and I'd like to use the appropriate one depending on the OS being analyzed.

I was hoping I could do something like that :

class MyPlugin(PluginInterface):

    @classmethod
    def get_requirements(cls):
        if os == "Windows":
            from volatility3.framework.plugins.windows.pslist import PsList
            kernel_req = ModuleRequirement(name = 'kernel', description = 'Windows kernel', architectures = ["Intel32", "Intel64"])

        elif os == "Linux":
            from volatility3.framework.plugins.linux.pslist import PsList
            kernel_req = ModuleRequirement(name = 'kernel', description = 'Linux kernel', architectures = ["Intel32", "Intel64"])
        else:
            raise NotImplemented

        return [
            kernel_req,
            ListRequirement(name = 'pid',
                            element_type = int,
                            description = "PIDs to include (all other processes are excluded)",
                            optional = True),

            PluginRequirement(name = 'pslist',
                              plugin = PsList,
                              version=(2, 0, 0))
        ]

But get_requirements() being a class method, I guess there are no way to retrieve context information from there. So what's the best way to achieve that ? Is there a way to call other plugins at runtime (by runtime I mean in the run() function) even if the plugin was not listed as a requirement ?

Any help would be greatly appreciated, thanks for reading me !

EDIT: is it even possible to have a common function to analyze an application on any OS ? I was hoping the memory layout would be close enough on all platforms to allow me do to that. Any tips on how to abstract that would be appreciated as well :)

ikelos commented 1 year ago

Hi there, so the reason for not combining everything is because scanning for all possible operating systems to find the right one can be time consuming and also may result in an image being classed as both windows as well as linux/mac or being classed as neither. The ModuleRequirement name isn't important, so you could just have one called 'kernel', no need to duplicate those. You'd need to depend on all the pslist plugins you'd want to use (the windows, linux and mac ones) since PsList in your example above is actually imported from a specific python module.

You'd then need some way to figure out which operating system volatility had found when it made your kernel, but the associated SymbolTable or possibly even the layer's metadata field might tell you, so that's probably not too difficult?

The trick of the plugins happening to have the same parameter names is coincidence, and will likely not hold up for all plugins, and you're right, you can't decide beforehand which parameters to ask for, because you haven't done anything with the image yet! That means you'll need to ask for all the possible parameters you may need and then figure out how to populate the configuration with the details you've got, that'll be the glue that joins the plugins together into one.

The memory layout is dependent on the architecture, so that is common for most of the operating systems but the structure of processes and the way they work/the information that's relevant to them, that's all OS specific. As such, I'm not sure I'd get your hopes up about being able to have one tool that automatically does everything you want?

Hope that helps answer your question?

ShellCode33 commented 1 year ago

Thanks for your answer ! That helps a lot

The ModuleRequirement name isn't important, so you could just have one called 'kernel', no need to duplicate those.

Of course... I must have been tired to not realise that the "description" attribute isn't mandatory... :sweat_smile:

I'm still having some trouble to find the OS detected by volatility though.

On both my Linux and Windows dumps, self.context.layers[kernel.layer_name].metadata["os"] returns Unknown.

So I tried to identify the OS through the context config, but still there's no clear distinction between OSes. I guess I can use plugins.MyPlugin.kernel.symbol_table_name.class but it seems a bit hackish, any additional recommendation regarding this ?

Here's the context config on my Windows 10 dump :

{ 
  "automagic.LayerStacker.single_location": "file:///home/shellcode/Tools/volatility3/windows10.dmp", 
  "automagic.LayerStacker.stackers": [ 
    "QemuStacker", 
    "Elf64Stacker", 
    "AVMLStacker", 
    "LimeStacker", 
    "WindowsCrashDumpStacker", 
    "VmwareStacker", 
    "MacIntelStacker", 
    "LinuxIntelStacker", 
    "WindowsIntelStacker" 
  ], 
  "plugins.MyPlugin.kernel": "kernel", 
  "plugins.MyPlugin.kernel.class": "volatility3.framework.contexts.Module", 
  "plugins.MyPlugin.kernel.layer_name": "layer_name", 
  "plugins.MyPlugin.kernel.layer_name.class": "volatility3.framework.layers.intel.WindowsIntel32e", 
  "plugins.MyPlugin.kernel.layer_name.kernel_virtual_offset": 272701973995520, 
  "plugins.MyPlugin.kernel.layer_name.memory_layer": "memory_layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer": "base_layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer.class": "volatility3.framework.layers.physical.FileLayer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer.location": "file:///home/shellcode/Tools/volatility3/windows10.dmp", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.class": "volatility3.framework.layers.elf.Elf64Layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.isf_url": "file:///home/shellcode/Tools/volatility3/volatility3/framework/symbols/linux/elf.json", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.symbol_mask": 0, 
  "plugins.MyPlugin.kernel.layer_name.page_map_offset": 1744896, 
  "plugins.MyPlugin.kernel.layer_name.swap_layers": true, 
  "plugins.MyPlugin.kernel.layer_name.swap_layers.number_of_elements": 0, 
  "plugins.MyPlugin.kernel.offset": 272701973995520, 
  "plugins.MyPlugin.kernel.symbol_table_name": "symbol_table_name1", 
  "plugins.MyPlugin.kernel.symbol_table_name.class": "volatility3.framework.symbols.windows.WindowsKernelIntermedSymbols", 
  "plugins.MyPlugin.kernel.symbol_table_name.isf_url": "file:///home/shellcode/Tools/volatility3/volatility3/symbols/windows/ntkrnlmp.pdb/8B11040A5928757B11390AC78F6B6925-1.json.xz", 
  "plugins.MyPlugin.kernel.symbol_table_name.symbol_mask": 0, 
  "plugins.MyPlugin.pid": [], 
  "plugins.MyPlugin.pslist": false, 
  "plugins.MyPlugin.pslist.pslist": true, 
  "plugins.MyPlugin.pslist2": false, 
  "plugins.MyPlugin.pslist2.pslist2": true, 
  "plugins.MyPlugin.pslist3": false, 
  "plugins.MyPlugin.pslist3.pslist3": true 
} 

And here's the context config of my Linux dump :

{ 
  "automagic.LayerStacker.single_location": "file:///home/shellcode/Tools/volatility3/Fedora37.dmp", 
  "automagic.LayerStacker.stackers": [ 
    "QemuStacker", 
    "Elf64Stacker", 
    "AVMLStacker", 
    "LimeStacker", 
    "WindowsCrashDumpStacker", 
    "VmwareStacker", 
    "MacIntelStacker", 
    "LinuxIntelStacker", 
    "WindowsIntelStacker" 
  ], 
  "plugins.MyPlugin.kernel": "kernel", 
  "plugins.MyPlugin.kernel.class": "volatility3.framework.contexts.Module", 
  "plugins.MyPlugin.kernel.layer_name": "layer_name", 
  "plugins.MyPlugin.kernel.layer_name.class": "volatility3.framework.layers.intel.Intel32e", 
  "plugins.MyPlugin.kernel.layer_name.kernel_banner": "Linux version 6.0.7-301.fc37.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 12.2.1 20220819 (Red Hat 12.2.1-2), GNU ld version 2.38-24.fc37) #1 SMP PREEMPT_DYNAMIC Fri Nov 4 18:35:48 UTC 2022\n\u0000", 
  "plugins.MyPlugin.kernel.layer_name.kernel_virtual_offset": 570425344, 
  "plugins.MyPlugin.kernel.layer_name.memory_layer": "memory_layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer": "base_layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer.class": "volatility3.framework.layers.physical.FileLayer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.base_layer.location": "file:///home/shellcode/Tools/volatility3/Fedora37.dmp", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.class": "volatility3.framework.layers.elf.Elf64Layer", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.isf_url": "file:///home/shellcode/Tools/volatility3/volatility3/framework/symbols/linux/elf.json", 
  "plugins.MyPlugin.kernel.layer_name.memory_layer.symbol_mask": 0, 
  "plugins.MyPlugin.kernel.layer_name.page_map_offset": 2063663104, 
  "plugins.MyPlugin.kernel.layer_name.swap_layers": true, 
  "plugins.MyPlugin.kernel.layer_name.swap_layers.number_of_elements": 0, 
  "plugins.MyPlugin.kernel.offset": 570425344, 
  "plugins.MyPlugin.kernel.symbol_table_name": "symbol_table_name1", 
  "plugins.MyPlugin.kernel.symbol_table_name.class": "volatility3.framework.symbols.linux.LinuxKernelIntermedSymbols", 
  "plugins.MyPlugin.kernel.symbol_table_name.isf_url": "file:///home/shellcode/Tools/volatility3/volatility3/symbols/linux/vmlinux-debug-fedora-kernel-6.0.7-301.json", 
  "plugins.MyPlugin.kernel.symbol_table_name.symbol_mask": 281474976710655, 
  "plugins.MyPlugin.pid": [], 
  "plugins.MyPlugin.pslist": false, 
  "plugins.MyPlugin.pslist.pslist": true, 
  "plugins.MyPlugin.pslist2": false, 
  "plugins.MyPlugin.pslist2.pslist2": true, 
  "plugins.MyPlugin.pslist3": false, 
  "plugins.MyPlugin.pslist3.pslist3": true 
} 
ikelos commented 1 year ago

The metadata should be applied to the layer (I believe it'll bubble upwards, so best to ask the highest layer) as can be seen:

https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/automagic/windows.py#L216 https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/automagic/linux.py#L92 https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/automagic/mac.py#L118

It is unfortunately not stored as a configuration variable (and you're welcome to file that as a bug if needed), it is a separate metadata attribute that can be requested from the layer. This means, the value may not be present on layers that are reconstructed from a config. The metadata feature isn't often used, other than by automagic, so at the moment there is no mechanism for this to be stored/restored from the config. It'll take a little thought to implement it, but it should be possible...