Windows 8: issues/discussions/etc (grouped list)

[GoogleCodeExporter]

When I was working on getting Windows 8 support for volatility, I came across a 
crash on pslist.  I just thought I'd log it here:

$ python vol.py -f WIN8_32-20110915-160043.raw --profile=Win8M3x86 pslist
Volatile Systems Volatility Framework 2.1_alpha
 Offset(V)  Name                 PID    PPID   Thds   Hnds   Time 
---------- -------------------- ------ ------ ------ ------ ------------------- 
Traceback (most recent call last):
  File "./vol.py", line 135, in <module>
    main()
  File "./vol.py", line 126, in main
    command.execute()
  File "Vol_Win8/volatility/commands.py", line 101, in execute
    func(outfd, data)
  File "Vol_Win8/volatility/plugins/taskmods.py", line 142, in render_text
    task.ObjectTable.HandleCount,
  File "Vol_Win8/volatility/obj.py", line 304, in __getattr__
    return getattr(proxied, attr)
AttributeError: 'int' object has no attribute 'HandleCount'

This is because _HANDLE_TABLE not longer has a "HandleCount" member.

Changing obj.py helped this:

Index: volatility/obj.py
===================================================================
--- volatility/obj.py   (revision 1074)
+++ volatility/obj.py   (working copy)
@@ -300,7 +300,10 @@
         if proxied is None:
             raise AttributeError("Unable to resolve attribute {0} on {1}".format(attr, self.obj_name))

-        return getattr(proxied, attr)
+        try:
+            return getattr(proxied, attr)
+        except AttributeError:
+            return NoneObject("No value for {0}".format(attr))

     def __setattr__(self, attr, value):
         try:

$ python vol.py -f WIN8_32-20110915-160043.raw --profile=Win8M3x86 pslist
Volatile Systems Volatility Framework 2.1_alpha
 Offset(V)  Name                 PID    PPID   Thds   Hnds   Time 
---------- -------------------- ------ ------ ------ ------ ------------------- 
0x8274aa00 System                    4      0     80 ------ 2011-09-15 06:31:31 

0x83967040 smss.exe                232      4      2 ------ 2011-09-15 06:31:31 

0x83fc2040 smss.exe                300    232      0 ------ 2011-09-15 06:31:32 

0x83fde940 csrss.exe               316    300      8 ------ 2011-09-15 06:31:33 

0x827fa780 smss.exe                372    232      0 ------ 2011-09-15 06:31:33 

0x82808040 csrss.exe               380    372      9 ------ 2011-09-15 06:31:33 

0x8279f640 wininit.exe             388    300      2 ------ 2011-09-15 06:31:33 

0x827f45c0 winlogon.exe            416    372      3 ------ 2011-09-15 06:31:33 

0x827cb040 services.exe            476    388      8 ------ 2011-09-15 06:31:34 

0x827a2780 WerFault.exe            484    388      0 ------ 2011-09-15 06:31:34 

0x83ffb580 lsass.exe               492    388      8 ------ 2011-09-15 06:31:34 

0x840bc040 svchost.exe             608    476      7 ------ 2011-09-15 06:31:36 

0x840c0d00 dwm.exe                 632    416      7 ------ 2011-09-15 06:31:36 

0x840cf4c0 svchost.exe             660    476     11 ------ 2011-09-15 06:31:36 

0x84084100 LogonUI.exe             760    416      0 ------ 2011-09-15 06:31:37 

0x841664c0 svchost.exe             772    476     23 ------ 2011-09-15 06:31:37 
[snip]

Though the handles are not there, obviously.... but at least it doesn't crash 
now.

Original issue reported on code.google.com by jamie.l...@gmail.com on 23 Sep 2011 at 1:31

[GoogleCodeExporter]

Mmmmm, we already raise AttributeErrors in obj.  It suggests that these should 
be caught around the calling function, rather than within obj.py.

Original comment by mike.auty@gmail.com on 25 Sep 2011 at 9:49

[GoogleCodeExporter]

Jamie do you have any references for how Win8 determines the total number of 
entries in the table? I would propose you add an overlay to win8 to put that 
HandleCount back into the _HANDLE_TABLE class to account for this new way.

Original comment by scude...@gmail.com on 28 Sep 2011 at 11:50

[GoogleCodeExporter]

Turning this issue about win8 missing attrib into a grouped list of issues 
regarding win8 and server2012. 

Its not a priority ATM but someone is passing me notes, so might as well log 
them here for when it does become important to us. 

* _HANDLE_TABLE_ENTRY.Object is no longer used, its 
_HANDLE_TABLE_ENTRY.ObjectPointerBits 
* _MMVAD_LONG is no longer a symbol in the PDB

Original comment by michael.hale@gmail.com on 10 Jul 2012 at 7:59

• Changed title: Windows 8: issues/discussions/etc (grouped list)
• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Just a quick comment about this issue. Its common that struct members change 
their names between versions and in the past we had to do an overlay for each 
specific version to proxy the names of members.

The current obj.py implementation also raises AttributeError when we access a 
non existing attribute in a CType which is a useful thing (to catch syntax 
errors etc). But sometimes we actually know that the attribute may or may not 
be defined depending on the profile. In this case its more useful to return a 
NoneObject() so we can get the attribute using a bunch of possibilities.

So I suggest to leave the AttributeError to be raised from __getattr__ but when 
calling the m() method we should return a NoneObject if the attribute does not 
exist. Then we can do something like:

object_pointer = table.m("Object") or table.m("ObjectPointerBits")

And indeed make an overlay such as:

overlay = {
  '_HANDLE_TABLE_ENTRY': [None, {
      'ObjectPointer': lambda x: x.m("Object") or x.m("ObjectPointerBits")
  }]
}

This overlay can therefore support both profiles at once.

Original comment by scude...@gmail.com on 10 Jul 2012 at 9:22

[GoogleCodeExporter]

Ok Just to add more information to this issue about windows 8.

The vad structures have changed in this release. The interesting thing is that 
the structures are nested, which means that lower level structs are always 
first in the layout of higher level structs. This is probably so that the code 
which is concerned with these can just cast the memory to the expected pointer 
type and does need to special case anything.

Now the main tree is constructed using _MM_AVL_NODE:

[_MM_AVL_NODE _MM_AVL_NODE] @ 0x00000000
  0x00 u1         [__unnamed_22b5 u1] @ 0x00000000
  0x08 LeftChild  <_MM_AVL_NODE Pointer to [0x00000000] (LeftChild)>
  0x10 RightChild <_MM_AVL_NODE Pointer to [0x00000000] (RightChild)>

The _MMVAD_SHORT has a first member (at offset 0):
  0x00 VadNode        [_MM_AVL_NODE VadNode] @ 0x00000000

The _MMVAD has a first member (at offset 0):
  0x00 Core              [_MMVAD_SHORT Core] @ 0x00000000

This means you can always cast any vad struct to a _MM_AVL_NODE to traverse it. 
Since in previous windows versions many of the fields had a different nesting. 
e.g. previously:

commit_charge = vad.u.VadFlags.CommitCharge

now:
commit_charge = vad.Core.u1.VadFlags1.CommitCharge

So we could just set up a bunch of redirectors/aliases in the profile overlays 
to obtain the stuff we care about in a profile independent way.

An example is:
http://code.google.com/p/volatility/source/browse/branches/scudette/volatility/p
lugins/overlays/windows/win8.py#49

Original comment by scude...@gmail.com on 16 Jul 2012 at 11:56

[GoogleCodeExporter]

Another topic with windows 8 is pool scanning.

The first thing we notice is that pool tags have changed. Most pool tags do not 
have non ascii data in them. For example tag for _EPROCESS used to be 'Pro\xe3' 
and it is now 'Proc'. We need therefore profile specific pool tags (Currently 
pool tags are hard coded in the pool scanners).

In previous version of windows we used a technique coined "bottom up" scanning 
to find the object header. The problem is that the object header can have 
multiple optional headers before it, and so going from the pool header to the 
object header is not simple:

Size
^   Pool Header
|   Optional Header1
|   Optional Header1
|   Object Header
|   Other Headers
v   Object data

The bottom up technique calculates the total size of the allocation, then goes 
backwards to find the object header by subtracting the size of the know objects 
in the allocation.

This method does not work with windows 8 since the allocations may get rounded 
up to the next bucket size. This means there is always some padding at the end 
of the objects. The padding may be random since the allocator may decide to 
give the next bucket size again.

This means that on windows 8 we need to use a more intensive scanning algorithm 
to determine the object header. Basically we check for optional headers and 
then instantiate an _OBJECT_HEADER object at a particular offset, and then we 
can check if the total size of the object headers matches what we expect.

This is more expensive since we need to check potentially about a dozen more 
position s per scan than before. Coupled with the fact that the pool tag is so 
common (e.g. 'Proc' matches a lot of stuff) this does tend to slow us down a 
bit. We need to add further scanning checks to try to eliminate false positives 
before we even get to the object header bit.

Sample implementation here:
http://code.google.com/p/volatility/source/browse/branches/scudette/volatility/p
lugins/overlays/windows/win8.py#193

This code changes the pool scanners to simply instantiate a _POOL_HEADER at the 
potential site, and then call _POOL_HEADER.get_object() to try and receive the 
object header. The specific implementation of the algorithm is then extracted 
to profile specific _POOL_HEADER implementations - the old windows use the old 
bottom up technique but win8 profiles use the new algorithm. The new algorithm 
also works for the old scheme too so an option is to get rid of the bottom up 
technique altogether.

Original comment by scude...@gmail.com on 16 Jul 2012 at 12:10

[GoogleCodeExporter]

Original comment by michael.hale@gmail.com on 1 Feb 2013 at 5:03

• Added labels: Milestone-3.0.x

[GoogleCodeExporter]

Hey guys - I'm going to close this down, as we have support for win8/2012 now. 

Original comment by michael.hale@gmail.com on 5 Nov 2013 at 7:39

• Changed state: Fixed