Linux.pslist dump : Page Fault

garanews commented 1 year ago

with vol -vvv -f /media/uploads/repo_ram_20200302.img linux.pslist --dump

getting this error:

Volatility 3 Framework 2.5.1

0x8802304e4600  1261    1261    1       accounts-daemon pid.1261.accounts-daemon.0x400000.dmp
0x8802314af000  1273    1273    1       agetty  pid.1273.agetty.0x400000.dmp
0x8802317b7000  1282    1282    1       polkitd pid.1282.polkitd.0x400000.dmp
0x8800bb8bf000  1458    1458    1       systemd pid.1458.systemd.0x55ee1b0fd000.dmp

DEBUG    volatility3.cli: Traceback (most recent call last):
  File "/src/volatility3/volatility3/cli/__init__.py", line 447, in run
    renderers[args.renderer]().render(constructed.run())
  File "/src/volatility3/volatility3/cli/text_renderer.py", line 193, in render
    grid.populate(visitor, outfd)
  File "/src/volatility3/volatility3/framework/renderers/__init__.py", line 241, in populate
    for level, item in self._generator:
  File "/src/volatility3/volatility3/framework/plugins/linux/pslist.py", line 125, in _generator
    file_handle = elfs.Elfs.elf_dump(
  File "/src/volatility3/volatility3/framework/plugins/linux/elfs.py", line 78, in elf_dump
    elf_object = context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/symbols/linux/extensions/elf.py", line 36, in __init__
    magic = self._context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 168, in __new__
    value = cls._unmarshall(context, data_format, object_info)
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 202, in _unmarshall
    data = context.layers.read(
  File "/src/volatility3/volatility3/framework/interfaces/layers.py", line 638, in read
    return self[layer].read(offset, length, pad)
  File "/src/volatility3/volatility3/framework/layers/linear.py", line 45, in read
    for offset, _, mapped_offset, mapped_length, layer in self.mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 283, in mapping
    for offset, size, mapped_offset, mapped_size, map_layer in self._mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 339, in _mapping
    chunk_offset, page_size, layer_name = self._translate(offset)
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 147, in _translate
    raise exceptions.PagedInvalidAddressException(
volatility3.framework.exceptions.PagedInvalidAddressException: Page Fault at entry 0x0 in page entry

Volatility was unable to read a requested page:
Page error 0x55c3f62d1000 in layer layer_name_Process1464 (Page Fault at entry 0x0 in page entry)

        * Memory smear during acquisition (try re-acquiring if possible)
        * An intentionally invalid page lookup (operating system protection)
        * A bug in the plugin/volatility3 (re-run with -vvv and file a bug)

No further results will be produced

Tried different dumps but result is same:

Volatility 3 Framework 2.5.1

DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nfs4_lock_state
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nlm_lockowner
Level 6  volatility3.framework.symbols.intermed: Searching for symbols in /src/volatility3/volatility3/symbols, /src/volatility3/volatility3/framework/symbols

DEBUG    volatility3.cli: Traceback (most recent call last):
  File "/src/volatility3/volatility3/cli/__init__.py", line 447, in run
    renderers[args.renderer]().render(constructed.run())
  File "/src/volatility3/volatility3/cli/text_renderer.py", line 193, in render
    grid.populate(visitor, outfd)
  File "/src/volatility3/volatility3/framework/renderers/__init__.py", line 241, in populate
    for level, item in self._generator:
  File "/src/volatility3/volatility3/framework/plugins/linux/pslist.py", line 125, in _generator
    file_handle = elfs.Elfs.elf_dump(
  File "/src/volatility3/volatility3/framework/plugins/linux/elfs.py", line 78, in elf_dump
    elf_object = context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/symbols/linux/extensions/elf.py", line 36, in __init__
    magic = self._context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 168, in __new__
    value = cls._unmarshall(context, data_format, object_info)
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 202, in _unmarshall
    data = context.layers.read(
  File "/src/volatility3/volatility3/framework/interfaces/layers.py", line 638, in read
    return self[layer].read(offset, length, pad)
  File "/src/volatility3/volatility3/framework/layers/linear.py", line 45, in read
    for offset, _, mapped_offset, mapped_length, layer in self.mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 283, in mapping
    for offset, size, mapped_offset, mapped_size, map_layer in self._mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 339, in _mapping
    chunk_offset, page_size, layer_name = self._translate(offset)
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 147, in _translate
    raise exceptions.PagedInvalidAddressException(
volatility3.framework.exceptions.PagedInvalidAddressException: Page Fault at entry 0x0 in page entry

Volatility was unable to read a requested page:
Page error 0x400000 in layer layer_name_Process1 (Page Fault at entry 0x0 in page entry)

        * Memory smear during acquisition (try re-acquiring if possible)
        * An intentionally invalid page lookup (operating system protection)
        * A bug in the plugin/volatility3 (re-run with -vvv and file a bug)

No further results will be produced

Another one: vol -vvvvvvv -f /media/7f1e93a6-f89f-11ed-81d7-0242ac17000a/linux-sample-2.bin linux.pslist --dump


DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nfs4_lock_state
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nlm_lockowner
Level 6  volatility3.framework.symbols.intermed: Searching for symbols in /src/volatility3/volatility3/symbols, /src/volatility3/volatility3/framework/symbols

DEBUG    volatility3.cli: Traceback (most recent call last):
  File "/src/volatility3/volatility3/cli/__init__.py", line 447, in run
    renderers[args.renderer]().render(constructed.run())
  File "/src/volatility3/volatility3/cli/text_renderer.py", line 193, in render
    grid.populate(visitor, outfd)
  File "/src/volatility3/volatility3/framework/renderers/__init__.py", line 241, in populate
    for level, item in self._generator:
  File "/src/volatility3/volatility3/framework/plugins/linux/pslist.py", line 125, in _generator
    file_handle = elfs.Elfs.elf_dump(
  File "/src/volatility3/volatility3/framework/plugins/linux/elfs.py", line 78, in elf_dump
    elf_object = context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/symbols/linux/extensions/elf.py", line 36, in __init__
    magic = self._context.object(
  File "/src/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/src/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 168, in __new__
    value = cls._unmarshall(context, data_format, object_info)
  File "/src/volatility3/volatility3/framework/objects/__init__.py", line 202, in _unmarshall
    data = context.layers.read(
  File "/src/volatility3/volatility3/framework/interfaces/layers.py", line 638, in read
    return self[layer].read(offset, length, pad)
  File "/src/volatility3/volatility3/framework/layers/linear.py", line 45, in read
    for offset, _, mapped_offset, mapped_length, layer in self.mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 283, in mapping
    for offset, size, mapped_offset, mapped_size, map_layer in self._mapping(
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 339, in _mapping
    chunk_offset, page_size, layer_name = self._translate(offset)
  File "/src/volatility3/volatility3/framework/layers/intel.py", line 147, in _translate
    raise exceptions.PagedInvalidAddressException(
volatility3.framework.exceptions.PagedInvalidAddressException: Page Fault at entry 0x0 in page entry

Volatility was unable to read a requested page:
Page error 0x400000 in layer layer_name_Process1 (Page Fault at entry 0x0 in page entry)

        * Memory smear during acquisition (try re-acquiring if possible)
        * An intentionally invalid page lookup (operating system protection)
        * A bug in the plugin/volatility3 (re-run with -vvv and file a bug)

No further results will be produced

So currently I have 100% of unsuccess :(

eve-mem commented 1 year ago

Hello there, from those logs it looks like the parts needed to correctly dump those processes are paged out (e.g not in ram) hence the errors.

Notice in the first run a few worked correctly.

It would be possible to pad this missing space with zeros, would that help as an extra command line option? That would let you extract what is available in ram (just know it wont be complete)

The correct solution would be adding swap file support for linux, which is on my todo list, just a fair amount of work. It's not always possible to get the swap either. I'm still working away trying to add the dump files side of things, and that feels more generically useful than swap.

eve-mem commented 1 year ago

Slight correction! It is still likely an issue where parts of memory aren't available but I'd assumed this was due to not using padded reads when building the elf, however that's not right. The main read is padded, so there's no need to add an extra option.

https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/plugins/linux/elfs.py#L118

I suspect it'll just need some extra protections to add, however it's not quite as straight forwarded as I'd assumed. Sorry!

eve-mem commented 1 year ago

@garanews could you give this branch a test? https://github.com/eve-mem/volatility3/tree/linux_elf_padded_magic_read

It should fix your issue, rather than crashing completely when reading the header it will now show Error outputting file. At least when it is the magic bytes that are missing, which I think is the issue with all of your samples - however there are other parts that may also fail so let me know if that happens.

Essentially the very first page of the elfs you were trying to save were missing, so it wasn't possible to check if they were actually elfs at all, and some other parts of the header are likely missing. You'd still be able to extract memory for these with linux.proc.Maps however it won't be stitched together into a nice elf file for you.

I hope this helps!

python vol.py -f linux-sample-2.dmp linux.pslist --pid 1 --dump
Volatility 3 Framework 2.5.1
Progress:  100.00               Stacking attempts finished
OFFSET (V)      PID     TID     PPID    COMM    File output

0x88001f994740  1       1       0       init    Error outputting file

garanews commented 1 year ago

Hello, I am getting same error: /elf/volatility3$ python vol.py -vvv -f /home/osboxes/volatility3/linux-sample-1.bin linux.pslist --dump

DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 3.2.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.57-3+deb7u2\n\x00'
DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: jar:file:/home/osboxes/elf/volatility3/volatility3/symbols/linux.zip!linux/linux-image-3.2.0-4-amd64-dbg_3.2.57-3+deb7u2_amd64.json.xz
INFO     volatility3.framework.automagic: Running automagic: KernelModule

OFFSET (V)      PID     TID     PPID    COMM    File output
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dma_coherent_mem
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nf_ct_event_notifier
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nf_exp_event_notifier
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!ebt_table
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!ip_vs_sync_buff
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_dev
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!garp_port
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phy_device
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!tcp_states_t
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_tstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_conn
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_cached_keys
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!cfg80211_internal_bss
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!libipw_device
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!dn_route
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nfs4_lock_state
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nlm_lockowner

DEBUG    volatility3.cli: Traceback (most recent call last):
  File "/home/osboxes/elf/volatility3/volatility3/cli/__init__.py", line 447, in run
    renderers[args.renderer]().render(constructed.run())
  File "/home/osboxes/elf/volatility3/volatility3/cli/text_renderer.py", line 193, in render
    grid.populate(visitor, outfd)
  File "/home/osboxes/elf/volatility3/volatility3/framework/renderers/__init__.py", line 241, in populate
    for level, item in self._generator:
  File "/home/osboxes/elf/volatility3/volatility3/framework/plugins/linux/pslist.py", line 125, in _generator
    file_handle = elfs.Elfs.elf_dump(
  File "/home/osboxes/elf/volatility3/volatility3/framework/plugins/linux/elfs.py", line 78, in elf_dump
    elf_object = context.object(
  File "/home/osboxes/elf/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/home/osboxes/elf/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/home/osboxes/elf/volatility3/volatility3/framework/symbols/linux/extensions/elf.py", line 36, in __init__
    magic = self._context.object(
  File "/home/osboxes/elf/volatility3/volatility3/framework/contexts/__init__.py", line 127, in object
    return object_template(
  File "/home/osboxes/elf/volatility3/volatility3/framework/objects/templates.py", line 96, in __call__
    return self.vol.object_class(
  File "/home/osboxes/elf/volatility3/volatility3/framework/objects/__init__.py", line 168, in __new__
    value = cls._unmarshall(context, data_format, object_info)
  File "/home/osboxes/elf/volatility3/volatility3/framework/objects/__init__.py", line 202, in _unmarshall
    data = context.layers.read(
  File "/home/osboxes/elf/volatility3/volatility3/framework/interfaces/layers.py", line 638, in read
    return self[layer].read(offset, length, pad)
  File "/home/osboxes/elf/volatility3/volatility3/framework/layers/linear.py", line 45, in read
    for offset, _, mapped_offset, mapped_length, layer in self.mapping(
  File "/home/osboxes/elf/volatility3/volatility3/framework/layers/intel.py", line 283, in mapping
    for offset, size, mapped_offset, mapped_size, map_layer in self._mapping(
  File "/home/osboxes/elf/volatility3/volatility3/framework/layers/intel.py", line 339, in _mapping
    chunk_offset, page_size, layer_name = self._translate(offset)
  File "/home/osboxes/elf/volatility3/volatility3/framework/layers/intel.py", line 147, in _translate
    raise exceptions.PagedInvalidAddressException(
volatility3.framework.exceptions.PagedInvalidAddressException: Page Fault at entry 0x0 in page entry

Volatility was unable to read a requested page:
Page error 0x400000 in layer layer_name_Process1 (Page Fault at entry 0x0 in page entry)

        * Memory smear during acquisition (try re-acquiring if possible)
        * An intentionally invalid page lookup (operating system protection)
        * A bug in the plugin/volatility3 (re-run with -vvv and file a bug)

No further results will be produced

eve-mem commented 1 year ago

@garanews - thanks for testing. Is the sha1 of the linux-sample-1.bin you are using 1c3a4627edca94a7ade3414592bef0e62d7d3bb6 ?

I've tested this on my version and it seemed to work for me.

Are you sure you are using the changes from that branch I'd linked to? The error here is showing the existing way of reading the elf magic which is the part I'd tried to change.

  File "/home/osboxes/elf/volatility3/volatility3/framework/symbols/linux/extensions/elf.py", line 36, in __init__
    magic = self._context.object(

If you look at volatility3/framework/symbols/linux/extensions/elf.py The line 36 should look like this:

magic = self._context.layers[layer_name].read(object_info.offset, 4, True)

Rather than the previous method which made an object, which is what that error is showing.

You can see the changes here if it helps?

https://github.com/volatilityfoundation/volatility3/compare/develop...eve-mem:volatility3:linux_elf_padded_magic_read?diff=unified

garanews commented 1 year ago

@eve-mem I confirm sha1, so we are using same file. :) About patched version, you right: I cloned your repo without switching the branch! It is working!| ^___^

volatilityfoundation / volatility3

Linux.pslist dump : Page Fault #1009