search

quentinleher / volatility

Automatically exported from code.google.com/p/volatility
0 stars 0 forks source link

trunk r1116 breaks apihooks #150

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. svn update to latest trunk (latest malware.py, too)
2. run apihooks module

imageinfo:
          Suggested Profile(s) : WinXPSP3x86, WinXPSP2x86 (Instantiated with WinXPSP2x86)
                     AS Layer1 : JKIA32PagedMemory (Kernel AS)
                     AS Layer2 : FileAddressSpace (****)
                      PAE type : No PAE
                           DTB : 0x39000
                          KDBG : 0x805532e0
                          KPCR : 0xffdff000
             KUSER_SHARED_DATA : 0xffdf0000
           Image date and time : 2011-10-05 01:46:40
     Image local date and time : 2011-10-05 01:46:40
          Number of Processors : 4
                    Image Type : Service Pack 3

svn export -r 1115 and problem goes away.. something in intel.py / 
legacyintel.py?

Original issue reported on code.google.com by dave.lau...@gmail.com on 7 Oct 2011 at 4:29

GoogleCodeExporter commented 8 years ago 
forgot to add the error msgs:

Traceback (most recent call last):
  File "****/vol.py", line 135, in <module>
    main()
  File "****/vol.py", line 126, in main
    command.execute()
  File "****/volatility/commands.py", line 101, in execute
    func(outfd, data)
  File "****/volatility/plugins/malware.py", line 2044, in render_text
    for (proc, type, current_mod, mod, func, src, dst, hooker, instruction) in data:
  File "****/volatility/plugins/malware.py", line 2028, in calculate
    for val in self.get_all_hooks(p, ps_ad, procs, mods, mod_addrs):
  File "****/volatility/plugins/malware.py", line 1979, in get_all_hooks
    for val in self.get_hooks(proc, space, mods, mod_addrs, mod, name):
  File "****/volatility/plugins/malware.py", line 1754, in get_hooks
    for dll, symbols in mod.imports():
  File "****/volatility/plugins/malware.py", line 551, in imports
    for dll, imp in desc.get_imports():
  File "****/volatility/plugins/malware.py", line 401, in get_imports
    if table[i].AddressOfData & IMAGE_ORDINAL_FLAG:
  File "****/volatility/obj.py", line 431, in method
    return method(*args, **kw)
TypeError: unsupported operand type(s) for &: 'NoneObject' and 'long'

Original comment by dave.lau...@gmail.com on 7 Oct 2011 at 4:50

GoogleCodeExporter commented 8 years ago 
Thanks I'll look into this (and let you know probably next week early).

Original comment by michael.hale@gmail.com on 7 Oct 2011 at 6:21

GoogleCodeExporter commented 8 years ago 
Here's a quick patch that may help determine what's causing the problem (by 
printing out the NoneObject's reason for being)...

Original comment by mike.auty@gmail.com on 7 Oct 2011 at 7:58

Attachments:

GoogleCodeExporter commented 8 years ago 
So this looks related to the recent address space support for pages in 
transition. It looks like this plugin is now finding more tables and some of 
these are not pages in this sample:

table[i].AddressOfData returns invalid memory NoneObject which apparently does 
not support the & operator. I suggest adding this method to NoneObject in 
volatility/obj.py:

def __and__(self, other):
   return False

Original comment by scude...@gmail.com on 7 Oct 2011 at 9:33

GoogleCodeExporter commented 8 years ago 
with patch applied:
TypeError: unsupported operand type(s) for &: 'NoneObject' and 'long'

with NoneObject modified:
TypeError: unsupported operand type(s) for +: 'int' and 'NoneObject'

Original comment by dave.lau...@gmail.com on 10 Oct 2011 at 1:48

GoogleCodeExporter commented 8 years ago 
Are there any other operators we should be adding whilst we add __and__?

Original comment by mike.auty@gmail.com on 16 Oct 2011 at 1:49

GoogleCodeExporter commented 8 years ago 
Mike,
  Try also adding the __add__ and __radd__ methods.  What is the backtrace for the error message (which is the line which triggers the invalid addition attempt?).

Original comment by scude...@gmail.com on 16 Oct 2011 at 10:06

GoogleCodeExporter commented 8 years ago 
Dave, could you please re-try with malware.py r127 
(http://code.google.com/p/malwarecookbook/source/browse/trunk/malware.py) and 
let me know how it goes? 

While the suggested patches for NoneObject would probably be a good thing for 
future cases, I found that I could add some other sanity checks to malware.py 
that make it back off before getting to the point where it uses 
"table[i].AddressOfData" (if it can predict that table[i].AddressOfData is 
going to be NoneObject). 

Thanks!

Original comment by michael.hale@gmail.com on 13 Nov 2011 at 6:44

GoogleCodeExporter commented 8 years ago 
hi, with malware.py r127 (and vol r1144):
Traceback (most recent call last):
  File "***/volatility/vol.py", line 135, in <module>
    main()
  File "***/volatility/vol.py", line 126, in main
    command.execute()
  File "***/volatility/volatility/commands.py", line 101, in execute
    func(outfd, data)
  File "***/volatility/volatility/plugins/malware.py", line 2106, in render_text
    for (proc, type, current_mod, mod, func, src, dst, hooker, instruction) in data:
  File "***/volatility/volatility/plugins/malware.py", line 2080, in calculate
    ps_ad = p.get_process_address_space()
  File "***/volatility/volatility/plugins/overlays/windows/windows.py", line 197, in get_process_address_space
    process_as = self.obj_vm.__class__(self.obj_vm.base, self.obj_vm.get_config(), dtb = directory_table_base)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 89, in __init__
    self.as_assert(getattr(volmag, self.checkname).v(), "Failed valid Address Space check")
  File "***/volatility/volatility/obj.py", line 801, in v
    return self.get_best_suggestion()
  File "***/volatility/volatility/obj.py", line 827, in get_best_suggestion
    for val in self.get_suggestions():
  File "***/volatility/volatility/obj.py", line 819, in get_suggestions
    for x in self.generate_suggestions():
  File "***/volatility/volatility/plugins/overlays/windows/windows.py", line 536, in generate_suggestions
    if (self.obj_vm.vtop(pde_base) == pd):
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 446, in vtop
    pte = self.get_pte(vaddr, pde)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 413, in get_pte
    return self._read_long_long_phys(pte_addr)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 458, in _read_long_long_phys
    string = self.base.read(addr, 8)
  File "***/volatility/volatility/plugins/addrspaces/standard.py", line 97, in read
    self.fhandle.seek(addr)
IOError: [Errno 22] Invalid argument

Original comment by dave.lau...@gmail.com on 14 Nov 2011 at 4:48

GoogleCodeExporter commented 8 years ago 
Wow, that's ugly. It looks however like an issue instantiating a process 
address space, which would be a framework-wide problem (not just apihooks). 

Thanks for your patience. If you don't mind, we could use the following data to 
help further troubleshoot:

1) can you run the attached plugin and paste the output from it?
2) do you get the same error on commands like vaddump and procexedump?

Thanks again!

Original comment by michael.hale@gmail.com on 14 Nov 2011 at 5:05

Attachments:

GoogleCodeExporter commented 8 years ago 
thanks, re 1):
Trying 4 System
... (hopefully ok that i snipped about ~72 procs from the output) ...
Trying 4720 cmd.exe
Trying 4452 fdpro.exe
Trying 5856 igfxsrvc.exe
Caught exception: [Errno 22] Invalid argument

perhaps interesting, the fail comes on the last process, and the only process 
listed after the mem dumper process..

0x87918ae8 cmd.exe                4720   2280      1     34 2011-10-10 19:52:47
0x878758d8 fdpro.exe              4452   4720      1     25 2011-10-10 19:53:01
0x87886da0 igfxsrvc.exe           5856    904      5      0 2011-10-10 19:53:03

re 2):
yup, if i procexedump or vaddump 5856 similar errors (other procs dump fine).
Traceback (most recent call last):
  File "***/volatility/vol.py", line 135, in <module>
    main()
  File "***/volatility/vol.py", line 126, in main
    command.execute()
  File "***/volatility/volatility/commands.py", line 101, in execute
    func(outfd, data)
  File "***/volatility/volatility/plugins/vadinfo.py", line 176, in render_text
    task_space = task.get_process_address_space()
  File "***/volatility/volatility/plugins/overlays/windows/windows.py", line 197, in get_process_address_space
    process_as = self.obj_vm.__class__(self.obj_vm.base, self.obj_vm.get_config(), dtb = directory_table_base)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 89, in __init__
    self.as_assert(getattr(volmag, self.checkname).v(), "Failed valid Address Space check")
  File "***/volatility/volatility/obj.py", line 801, in v
    return self.get_best_suggestion()
  File "***/volatility/volatility/obj.py", line 827, in get_best_suggestion
    for val in self.get_suggestions():
  File "***/volatility/volatility/obj.py", line 819, in get_suggestions
    for x in self.generate_suggestions():
  File "***/volatility/volatility/plugins/overlays/windows/windows.py", line 536, in generate_suggestions
    if (self.obj_vm.vtop(pde_base) == pd):
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 446, in vtop
    pte = self.get_pte(vaddr, pde)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 413, in get_pte
    return self._read_long_long_phys(pte_addr)
  File "***/volatility/volatility/plugins/addrspaces/intel.py", line 458, in _read_long_long_phys
    string = self.base.read(addr, 8)
  File "***/volatility/volatility/plugins/addrspaces/standard.py", line 97, in read
    self.fhandle.seek(addr)
IOError: [Errno 22] Invalid argument

Original comment by dave.lau...@gmail.com on 14 Nov 2011 at 5:37

GoogleCodeExporter commented 8 years ago 
Well that confirms the issue is part of volatility core. For other devs 
(Ikelos, Scudette, etc) this is very similar to the other open issue:

http://code.google.com/p/volatility/issues/detail?id=146 (fatal exception 
acquiring an address space (vaddump, memdump)

I bumped the issue to critical since it affects multiple plugins in the main 
trunk. 

Dave, I know its highly unlikely that you can share the memory dump, but if you 
can that would be cool. Guys, would any other tests or data be useful here?

Original comment by michael.hale@gmail.com on 14 Nov 2011 at 5:58

GoogleCodeExporter commented 8 years ago 
as you guessed i can't share this particular image. ...let me see if i can 
replicate on a lab box. i seem to recall seeing this error a few times. if you 
want outputs or something run let me know. thx!

Original comment by dave.lau...@gmail.com on 14 Nov 2011 at 8:01

GoogleCodeExporter commented 8 years ago 
Please try to apply the following patch and report back. Thanks.

$ svn diff branches/lin64-support/volatility/plugins/addrspaces/intel.py
Index: branches/lin64-support/volatility/plugins/addrspaces/intel.py
===================================================================
--- branches/lin64-support/volatility/plugins/addrspaces/intel.py       
(revision 1153)
+++ branches/lin64-support/volatility/plugins/addrspaces/intel.py       
(working copy)
@@ -380,8 +380,10 @@
         if self.cache:
             return self.pdpte_cache[self.pdpte_index(vaddr)]

-        pdpte_addr = (self.dtb & 0xffffffe0) | ((vaddr & 0xc0000000) >> 27)
-        return self._read_long_long_phys(pdpte_addr)
+        try:
+            pdpte_addr = (self.dtb & 0xffffffe0) | ((vaddr & 0xc0000000) >> 27)
+            return self._read_long_long_phys(pdpte_addr)
+        except IOError: pass

     def get_pde(self, vaddr, pdpte):
         '''
@@ -392,10 +394,11 @@
         Bits 11:3 are bits 29:21 of the linear address
         Bits 2:0 are 0
         '''
-        pde_addr = (pdpte & 0xffffffffff000) | ((vaddr & 0x3fe00000) >> 18)
-        return self._read_long_long_phys(pde_addr)
+        try:
+            pde_addr = (pdpte & 0xffffffffff000) | ((vaddr & 0x3fe00000) >> 18)
+            return self._read_long_long_phys(pde_addr)
+        except IOError: pass

-
     def get_two_meg_paddr(self, vaddr, pde):
         '''
         Return the offset in a 2MB memory page from the given virtual
@@ -415,8 +418,11 @@
         Bits 11:3 are bits 20:12 of the original linear address
         Bits 2:0 are 0
         '''
-        pte_addr = (pde & 0xffffffffff000) | ((vaddr & 0x1ff000) >> 9)
-        return self._read_long_long_phys(pte_addr)
+        try:
+            pte_addr = (pde & 0xffffffffff000) | ((vaddr & 0x1ff000) >> 9)
+            return self._read_long_long_phys(pte_addr)
+        except IOError:
+            return None

     def get_phys_addr(self, vaddr, pte):

Original comment by scude...@google.com on 11 Dec 2011 at 3:11

GoogleCodeExporter commented 8 years ago 
Here's another report of what seems to be the same issue:

What steps will reproduce the problem?
1.vol.py --plugins=/usr/local/src/volatility-2.0/volatility/plugins -f 
memory_dump.raw --profile=WinXPSP3x86 malfind -D malfind/ > malfind.out
2.
3.

What is the expected output? What do you see instead?
I get at least partial output, in that some sections are dumped, but this 
particular image generates the below error.

What version of the product are you using? On what operating system?
# uname -a
Linux aardvark 2.6.32-32-generic #62-Ubuntu SMP Wed Apr 20 21:52:38 UTC 2011 
x86_64 GNU/Linux

malware.py was installed on Nov. 18th...

Image was taken with Helix 2009 R3 live CD

Please provide any additional information below.

# vol.py --plugins=/usr/local/src/volatility-2.0/volatility/plugins -f 
memory_dump.raw --profile=WinXPSP3x86 malfind -D malfind/ > malfind.out
Volatile Systems Volatility Framework 2.0
Traceback (most recent call last):
 File "/usr/local/bin/vol.py", line 135, in <module>
   main()
 File "/usr/local/bin/vol.py", line 126, in main
   command.execute()
 File "/usr/local/lib/python2.6/dist-packages/volatility/commands.py", line 101, in execute
   func(outfd, data)
 File "/usr/local/src/volatility-2.0/volatility/plugins/malware.py", line 1042, in render_text
   for (name,pid,start,end,tag,prx,fname,hits,chunk) in data:
 File "/usr/local/src/volatility-2.0/volatility/plugins/malware.py", line 992, in calculate
   for ps_ad, start, end, tag, prx, data in self.get_vads(proc):
 File "/usr/local/src/volatility-2.0/volatility/plugins/malware.py", line 909, in get_vads
   ps_ad = proc.get_process_address_space()
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/overlays/windows/windows.py", line 197, in get_process_address_space
   process_as = self.obj_vm.__class__(self.obj_vm.base, self.obj_vm.get_config(), dtb = directory_table_base)
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/addrspaces/intel.py", line 89, in __init__
   self.as_assert(getattr(volmag, checkname).v(), "Failed valid Address Space check")
 File "/usr/local/lib/python2.6/dist-packages/volatility/obj.py", line 801, in v
   return self.get_best_suggestion()
 File "/usr/local/lib/python2.6/dist-packages/volatility/obj.py", line 827, in get_best_suggestion
   for val in self.get_suggestions():
 File "/usr/local/lib/python2.6/dist-packages/volatility/obj.py", line 819, in get_suggestions
   for x in self.generate_suggestions():
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/overlays/windows/windows.py", line 505, in generate_suggestions
   if (self.obj_vm.vtop(0xffdf0000)) == (self.obj_vm.vtop(0x7ffe0000)):
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/addrspaces/intel.py", line 447, in vtop
   pte = self.get_pte(vaddr, pde)
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/addrspaces/intel.py", line 414, in get_pte
   return self._read_long_long_phys(pte_addr)
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/addrspaces/intel.py", line 459, in _read_long_long_phys
   string = self.base.read(addr, 8)
 File "/usr/local/lib/python2.6/dist-packages/volatility/plugins/addrspaces/standard.py", line 97, in read
   self.fhandle.seek(addr)
IOError: [Errno 22] Invalid argument

Original comment by michael.hale@gmail.com on 16 Dec 2011 at 8:09

GoogleCodeExporter commented 8 years ago 
I asked Jesse to apply scudette's patch in comment 14 and he said it fixed the 
issue presented in comment 15 (well it prevented the exceptions & stack dump 
anyway). 

So that is good news once we finalize the patch to use NoneObject (I think 
that's what we discussed on the dev call) and also applying it to the legacy 
intel address space.

Original comment by michael.hale@gmail.com on 16 Dec 2011 at 11:25

GoogleCodeExporter commented 8 years ago 
nice, patch applied and everything looks swell. apihooks doesn't crash any 
longer. similarly, the plugin provided at comment 10 doesn't catch any 
exceptions either. thanks!

Original comment by dave.lau...@gmail.com on 19 Dec 2011 at 4:37

GoogleCodeExporter commented 8 years ago 
Hi Dave, thanks very much testing that.  A very similar patch has now been 
applied (in r1165).  Please could you test out the latest trunk and let me know 
if this has the same effect?

Original comment by mike.auty@gmail.com on 19 Dec 2011 at 11:07

GoogleCodeExporter commented 8 years ago 
hey - at revision 1165, and all is still looking swell.

Original comment by dave.lau...@gmail.com on 20 Dec 2011 at 3:06

GoogleCodeExporter commented 8 years ago 
Thanks very much for double checking that Dave.  As such, I'm going to mark 
this as Fixed.

Original comment by mike.auty@gmail.com on 20 Dec 2011 at 3:09

GoogleCodeExporter commented 8 years ago 
thanks!

Original comment by dave.lau...@gmail.com on 20 Dec 2011 at 3:15