search

volatilityfoundation / profiles

Volatility profiles for Linux and Mac OS X
317 stars 101 forks source link

Is it truly possible to use Volatility with Linux memory dumps? #46

Closed cpuu closed 6 years ago

cpuu commented 7 years ago

(I'm sorry I can not write English well)

Have you guys ever used Volatility Framework with Linux memory dumps, of recent day ?

I have been working hard for a few days.

I think that Volatility is the de facto standard in window analysis, on the other hands, it does not work with Memdump from Linux.

Recent kernel versions appear to have broken compatibility. Even if you use some profiles provided by official github, it doesn't match even if one number is different by 3 decimal points.

I Used LiME and lmg(linux memory grabber) for creating profile and dumping physical memory. Everything is OK.

but, Volatility cannot parse the data. I tried CentOS, Ubuntu, Kali Linux, Debian, Fedora .. and so on and so forth.

Every case, this messages shown

No suitable address space mapping found Tried to open image as: MachOAddressSpace: mac: need base LimeAddressSpace: lime: need base WindowsHiberFileSpace32: No base Address Space WindowsCrashDumpSpace64BitMap: No base Address Space WindowsCrashDumpSpace64: No base Address Space HPAKAddressSpace: No base Address Space VMWareMetaAddressSpace: No base Address Space VirtualBoxCoreDumpElf64: No base Address Space VMWareAddressSpace: No base Address Space QemuCoreDumpElf: No base Address Space WindowsCrashDumpSpace32: No base Address Space Win10AMD64PagedMemory: No base Address Space WindowsAMD64PagedMemory: No base Address Space LinuxAMD64PagedMemory: No base Address Space AMD64PagedMemory: No base Address Space IA32PagedMemoryPae: No base Address Space IA32PagedMemory: No base Address Space OSXPmemELF: No base Address Space MachOAddressSpace: MachO Header signature invalid MachOAddressSpace: MachO Header signature invalid LimeAddressSpace: Invalid Lime header signature WindowsHiberFileSpace32: PO_MEMORY_IMAGE is not available in profile WindowsCrashDumpSpace64BitMap: Header signature invalid WindowsCrashDumpSpace64: Header signature invalid HPAKAddressSpace: Invalid magic found VMWareMetaAddressSpace: VMware metadata file is not available VirtualBoxCoreDumpElf64: ELF Header signature invalid VMWareAddressSpace: Invalid VMware signature: 0xee300 QemuCoreDumpElf: ELF Header signature invalid WindowsCrashDumpSpace32: Header signature invalid Win10AMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected WindowsAMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected LinuxAMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected AMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected IA32PagedMemoryPae: Failed valid Address Space check IA32PagedMemory: Failed valid Address Space check OSXPmemELF: ELF Header signature invalid FileAddressSpace: Must be first Address Space ArmAddressSpace: Failed valid Address Space check

— Was there a mistake in my work? Or does the volatility still not support those versions of Linux?

deeso commented 7 years ago

What command are you using and are the profies in your path? Does volatility see the profiles in your path?

On May 1, 2017 6:48 AM, "cpuu" notifications@github.com wrote:

(I'm sorry I can not write English well)

Have you guys ever used Volatility Framework with Linux memory dumps, of recent day ?

I have been working hard for a few days.

I think that Volatility is the de facto standard in window analysis, on the other hands, it does not work with Memdump from Linux.

Recent kernel versions appear to have broken compatibility. Even if you use some profiles provided by official github, only one number differs three decimal places, it does not match.

I Used LiME and lmg(linux memory grabber) for creating profile and dumping physical memory. Everything is OK.

but, Volatility cannot parse the data. I tried CentOS, Ubuntu, Kali Linux, Debian, Fedora .. and so on and so forth.

Every case, this messages shown

Offset Name Pid PPid Uid Gid DTB Start Time

No suitable address space mapping found Tried to open image as: MachOAddressSpace: mac: need base LimeAddressSpace: lime: need base WindowsHiberFileSpace32: No base Address Space WindowsCrashDumpSpace64BitMap: No base Address Space WindowsCrashDumpSpace64: No base Address Space HPAKAddressSpace: No base Address Space VMWareMetaAddressSpace: No base Address Space VirtualBoxCoreDumpElf64: No base Address Space VMWareAddressSpace: No base Address Space QemuCoreDumpElf: No base Address Space WindowsCrashDumpSpace32: No base Address Space Win10AMD64PagedMemory: No base Address Space WindowsAMD64PagedMemory: No base Address Space LinuxAMD64PagedMemory: No base Address Space AMD64PagedMemory: No base Address Space IA32PagedMemoryPae: No base Address Space IA32PagedMemory: No base Address Space OSXPmemELF: No base Address Space MachOAddressSpace: MachO Header signature invalid MachOAddressSpace: MachO Header signature invalid LimeAddressSpace: Invalid Lime header signature WindowsHiberFileSpace32: PO_MEMORY_IMAGE is not available in profile WindowsCrashDumpSpace64BitMap: Header signature invalid WindowsCrashDumpSpace64: Header signature invalid HPAKAddressSpace: Invalid magic found VMWareMetaAddressSpace: VMware metadata file is not available VirtualBoxCoreDumpElf64: ELF Header signature invalid VMWareAddressSpace: Invalid VMware signature: 0xee300 QemuCoreDumpElf: ELF Header signature invalid WindowsCrashDumpSpace32: Header signature invalid Win10AMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected WindowsAMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected LinuxAMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected AMD64PagedMemory: Incompatible profile Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 selected IA32PagedMemoryPae: Failed valid Address Space check IA32PagedMemory: Failed valid Address Space check OSXPmemELF: ELF Header signature invalid FileAddressSpace: Must be first Address Space ArmAddressSpace: Failed valid Address Space check

— Was there a mistake in my work? Or does the volatility still not support those versions of Linux?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/volatilityfoundation/profiles/issues/46, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJkL8IQeZiqbVmfiPNHQlMytKeWihrhks5r1cZ0gaJpZM4NM_mD .

cpuu commented 7 years ago

sure, of course, I tried hundreds of times, with tens of kernels(Ubuntu, CentOS, Fedora.. )

volatility 2.6 osboxes@osboxes:~/volatility$ python vol.py --info Volatility Foundation Volatility Framework 2.6

bold is official profile and others are mine

Profiles

LinuxKali-Linux-2017x64 - A Profile for Linux Kali-Linux-2017 x64 LinuxMyUbuntu1604x64 - A Profile for Linux MyUbuntu1604 x64 LinuxUbuntu16041x64 - A Profile for Linux Ubuntu16041 x64 LinuxUbuntu1604x64 - A Profile for Linux Ubuntu1604 x64 Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 - A Profile for Linux cpuu-VirtualBox-2017-05-01_04.31.48-profile x86 Linuxosboxes-2017-05-01_07_01_18-profilex86 - A Profile for Linux osboxes-2017-05-01_07.01.18-profile x86 Linuxosboxes-2017-05-01_21_43_17-profilex64 - A Profile for Linux osboxes-2017-05-01_21.43.17-profile x64 Linuxsiftworkstation-2017-05-01_10_06_25-profilex64 - A Profile for Linux siftworkstation-2017-05-01_10.06.25-profile x64

Alright ..

I used linux_pslist , linux_psaux and so on. none of them effect.

Does it cause VirtualBox? why it not works?

deeso commented 7 years ago

When you execute the volatility command, do you specify the profile for the memory image?

On May 1, 2017 7:59 AM, "cpuu" notifications@github.com wrote:

sure, of course, I tried hundreds of times, with tens of kernels(Ubuntu, CentOS, Fedora.. )

volatility 2.6 osboxes@osboxes:~/volatility$ python vol.py --info Volatility Foundation Volatility Framework 2.6 Profiles

LinuxKali-Linux-2017x64 - A Profile for Linux Kali-Linux-2017 x64 LinuxMyUbuntu1604x64 - A Profile for Linux MyUbuntu1604 x64 LinuxUbuntu16041x64 - A Profile for Linux Ubuntu16041 x64 LinuxUbuntu1604x64 - A Profile for Linux Ubuntu1604 x64 Linuxcpuu-VirtualBox-2017-05-01_04_31_48-profilex86 - A Profile for Linux cpuu-VirtualBox-2017-05-01_04.31.48-profile x86 Linuxosboxes-2017-05-01_07_01_18-profilex86 - A Profile for Linux osboxes-2017-05-01_07.01.18-profile x86 Linuxosboxes-2017-05-01_21_43_17-profilex64 - A Profile for Linux osboxes-2017-05-01_21.43.17-profile x64 Linuxsiftworkstation-2017-05-01_10_06_25-profilex64 - A Profile for Linux siftworkstation-2017-05-01_10.06.25-profile x64

Alright ..

I used linux_pslist , linux_psaux and so on. none of them effect.

Does it cause VirtualBox? why it not works?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/volatilityfoundation/profiles/issues/46#issuecomment-298329842, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJkL5fQNHMHCLlwxkS5mI3hBbG2Seopks5r1dcwgaJpZM4NM_mD .

cpuu commented 7 years ago

sure, of course. I am doing well on Windows cases.. but why it does not show the results only in Linux.. I don't know what to do

deeso commented 7 years ago

I meant to ask for the specific command you are running. I am trying to set context and identify the root cause of the problem. Can you share the exact command you are running?

cpuu commented 7 years ago

When I use lmg usb style : sudo python vol.py --conf-file=../capture/osboxes-2017-05-01_21.43.17/volatilityrc linux_bash | head sudo python vol.py --conf-file=../capture/osboxes-2017-05-01_21.43.17/volatilityrc linux_psaux in volatilityrc file : cpuu@osboxes:/media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17$ cat volatilityrc [DEFAULT] PLUGINS=/media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17 PROFILE=Linuxosboxes-2017-05-01_21_43_17-profilex64 LOCATION=file:////media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17/osboxes-2017-05-01_21.43.17-memory.lime

but it not works!

So I copied the files on local system volatility dir, profile is in /home/osboxes/volatility/volatility/plugins/overlays/linux

osboxes@osboxes:~/volatility$ python vol.py --info shows me below very well. profiles : Linuxosboxes-2017-05-01_21_43_17-profilex64 - A Profile for Linux osboxes-2017-05-01_21.43.17-profile x64

now that I try ..

sudo python vol.py -f ../Desktop/osboxes-2017-05-01_21.43.17-memory.lime --profile=Linuxosboxes-2017-05-01_21_43_17-profilex64 linux_pslist

not works!

deeso commented 7 years ago

Ok. Could it be an issue with lime? Have you tried loading a memory snap shot from Vbox?

On May 1, 2017 8:26 AM, "cpuu" notifications@github.com wrote:

Normal case : sudo python vol.py -f ../Desktop/osboxes-2017-05-01_07.01.18-memory.lime --profile=Linuxosboxes-2017-05-01_07_01_18-profilex86 linux_pslist

When I use lmg https://github.com/halpomeranz/lmg style : sudo python vol.py --conf-file=../capture/osboxes-2017-05-01_21.43.17/volatilityrc linux_bash | head

in volatilityrc file : cpuu@osboxes:/media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17$ cat volatilityrc [DEFAULT] PLUGINS=/media/cpuu/ LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17 PROFILE=Linuxosboxes-2017-05-01_21_43_17-profilex64 LOCATION=file:////media/cpuu/LinuxMemoryGrab/lmg/capture/ osboxes-2017-05-01_21.43.17/osboxes-2017-05-01_21.43.17-memory.lime

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/volatilityfoundation/profiles/issues/46#issuecomment-298333107, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJkL5PVCHocjY3S3sX9thqtux_5HG62ks5r1d1ngaJpZM4NM_mD .

cpuu commented 7 years ago

Oh I have not heard about that. I will search that way and try it/

atcuno commented 7 years ago

Can you list uname -a for the system where you ran lmg? Also, what is the distro? And is this a virtual machine or a physical system?

Thanks, Andrew (@attrc)

On 05/01/2017 08:25 AM, cpuu wrote:

Normal case : sudo python vol.py -f ../Desktop/osboxes-2017-05-01_07.01.18-memory.lime --profile=Linuxosboxes-2017-05-01_07_01_18-profilex86 linux_pslist

When I use lmg style : sudo python vol.py --conf-file=../capture/osboxes-2017-05-01_21.43.17/volatilityrc linux_bash | head

in volatilityrc file : cpuu@osboxes:/media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17$ cat volatilityrc [DEFAULT] PLUGINS=/media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17 PROFILE=Linuxosboxes-2017-05-01_21_43_17-profilex64 LOCATION=file:////media/cpuu/LinuxMemoryGrab/lmg/capture/osboxes-2017-05-01_21.43.17/osboxes-2017-05-01_21.43.17-memory.lime

cpuu commented 7 years ago

I tried Ubuntu, Fedora, CentOS and so on.

especially, choose one distro, Ubuntu 16.04.1

uname -a Linux osboxes 4.8.0-49-generic #52~16.04.1-Ubuntu SMP Thu Apr 20 10:55:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux This is virtualbox for windows version 5.0.26 r108824

When I did it on the real Linux machine . Linux sep_gpu1 4.2.0-38-generic #45~14.04.1-Ubuntu SMP Thu Jun 9 09:27:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Both cases (each case has memdump itself) do not work.

All case was dumped by LiME. I will try using other dumper (/dev/kmem ? ) and report them soon.

cpuu commented 7 years ago

I did with VMware Workstation (not Virtual Box) on the 4.4.0-31-generic Ubuntu 14.04

python vol.py --conf-file=../capture/ubuntu-2017-05-02_08.36.09/volatilityrc linux_pslist Volatility Foundation Volatility Framework 2.6 Offset Name Pid PPid Uid Gid DTB Start Time

0xffff88003cc60000 init 1 0 0 0 0x000000003cac1000 2017-05-02 15:27:55 UTC+0000 0xffff88003cc60dc0 kthreadd 2 0 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc61b80 ksoftirqd/0 3 2 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc62940 kworker/0:0 4 2 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc63700 kworker/0:0H 5 2 0 0 ------------------ 2017-05-02 15:27:55

it works!!! oh shit... My effort was wasted... is it virtual box flaw ? i have no idea. but i did!!

deeso commented 7 years ago

I have used Linux KVM in the past without any issues..

On May 2, 2017 10:42 AM, "cpuu" notifications@github.com wrote:

I did with VMware Workstation (not Virtual Box) on the 4.4.0-31-generic Ubuntu 14.04

python vol.py --conf-file=../capture/ubuntu-2017-05-02_08.36.09/volatilityrc linux_pslist Volatility Foundation Volatility Framework 2.6 Offset Name Pid PPid Uid Gid DTB Start Time

0xffff88003cc60000 init 1 0 0 0 0x000000003cac1000 2017-05-02 15:27:55 UTC+0000 0xffff88003cc60dc0 kthreadd 2 0 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc61b80 ksoftirqd/0 3 2 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc62940 kworker/0:0 4 2 0 0 ------------------ 2017-05-02 15:27:55 UTC+0000 0xffff88003cc63700 kworker/0:0H 5 2 0 0 ------------------ 2017-05-02 15:27:55

it works!!! oh shit... My effort was wasted... is it virtual box flaw ? i have no idea. but i did!!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/volatilityfoundation/profiles/issues/46#issuecomment-298673908, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJkL-1LqFx0yrtJ9jB1AQYlz4cHM2_bks5r107NgaJpZM4NM_mD .

bneuburg commented 7 years ago

If you are using LiME to acquire memory try passing the timeout=0 module parameter when modprobing. This bit me a few times when trying to dump the memory from a KVM virtual machine. It's possible that VirtualBox is too slow in returning some of the memory pages and in this case LiME will just write zeroes instead of waiting.