search

Leor3961 / volatility

Automatically exported from code.google.com/p/volatility
0 stars 0 forks source link

Task.handles doesn't return all the handles that volatility-1.3.2 returned #27

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
I am missing output for the files command on some PIDs here is a sample:

$ ./volatility.py files -f ~/Work/volatility/xpsp3.vmem profile=WinXPSP3
Volatile Systems Volatility Framework 1.4_rc1
Pid:      4
File   \pagefile.sys                           
File   \WINDOWS\system32\config\default        
File   \System Volume 
Information\_restore{57726F20-9F0F-4D22-B53C-B6F197BAA74A}\RP72\change.log
File   \Topology                               
************************************************************************

now for 1.3.2:

$ ./volatility files -f /Users/gleeda/Work/volatility/xpsp3.vmem

************************************************************************
Pid: 4    
File   \pagefile.sys    
File   \WINDOWS\system32\config\default    
File   \System Volume 
Information\_restore{57726F20-9F0F-4D22-B53C-B6F197BAA74A}\RP72\change.log
File   \WINDOWS\system32\config\system    
File   \WINDOWS\system32\config\SECURITY    
File   \WINDOWS\system32\config\SECURITY.LOG   
File   \WINDOWS\system32\config\software    
File   \WINDOWS\system32\config\software.LOG   
File   \WINDOWS\system32\config\system.LOG    
File   \WINDOWS\system32\config\default.LOG    
File   \WINDOWS\system32\config\SAM    
File   \WINDOWS\system32\config\SAM.LOG    
File   \Documents and Settings\NetworkService\ntuser.dat.LOG
File   \Documents and Settings\NetworkService\NTUSER.DAT
File   \Documents and Settings\NetworkService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat
File   \Documents and Settings\NetworkService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat.LOG
File   \Documents and Settings\LocalService\ntuser.dat.LOG
File   \Documents and Settings\LocalService\NTUSER.DAT
File   \Documents and Settings\LocalService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat
File   \Documents and Settings\LocalService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat.LOG
File   \Documents and Settings\Administrator\NTUSER.DAT
File   \Documents and Settings\Administrator\ntuser.dat.LOG
File   \Documents and Settings\Administrator\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat
File   \Documents and Settings\Administrator\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat.LOG
File   \255    
File   \WINDOWS\Temp\hlktmp    
File   \    
File   \47    
File   \Topology    
************************************************************************

I get the same with or without "profile=WinXPSP3"

Original issue reported on code.google.com by jamie.l...@gmail.com on 29 Aug 2010 at 5:17

GoogleCodeExporter commented 8 years ago 
i meant "--profile=WinXPSP3"  still get the same results though.

Original comment by jamie.l...@gmail.com on 29 Aug 2010 at 5:24

GoogleCodeExporter commented 8 years ago 
I did a small test on an XPSP2 image, and sure enough they didn't turn up the 
same results.

Looking at the code paths involved, 1.3.2 uses a single function, unrolled, to 
look through the various tables of tables.  Those that aren't an end-node got 
an array size of 0x400, those that were of only 0x200.  In the 1.4 code, a 
recursive function was used, and each stage was only given an array of 0x200.  
I increased the size of arrays with a table_level > 0, and turned up some 
additional results, but *not* those missing from the 1.3.2 output.  I then 
tried using 0x400 for all of the arrays, and again got a few more results, but 
the 1.3.2 output still had some that weren't in the 1.4 output.

So really I need to call in someone who knows how these ObjectTables work, and 
see if they can figure out what's missing and why.  I'll continue having a poke 
around and see if I can pull back those remaining entries, but it's really 
unclear as to what's going on (and why it doesn't happen all the time).

For those that want it, here's the quick change to 1.4 to get additional 
results.

Original comment by mike.auty@gmail.com on 2 Sep 2010 at 4:43

Attachments:

GoogleCodeExporter commented 8 years ago 
I tried the same fix on my own earlier, but it crashes on the XP SP3 image 
above:

$ ./volatility.py files -f ~/Work/volatility/xpsp3.vmem 
Volatile Systems Volatility Framework 1.4_rc1
Pid:      4
File   \pagefile.sys                           
File   \WINDOWS\system32\config\default        
File   \System Volume 
Information\_restore{57726F20-9F0F-4D22-B53C-B6F197BAA74A}\RP72\change.log
File   \Topology                               
Traceback (most recent call last):
  File "./volatility.py", line 138, in <module>
    main()
  File "./volatility.py", line 129, in main
    command.execute()
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/commands.py", line 96, in execute
    func(outfd, data)
  File "/Users/gleeda/Work/TESTING/vol_1.4/plugins/internal/taskmods.py", line 123, in render_text
    for h in handles:
  File "/Users/gleeda/Work/TESTING/vol_1.4/plugins/internal/taskmods.py", line 139, in handle_list
    for h in task.handles():
  File "/Users/gleeda/Work/TESTING/vol_1.4/plugins/overlays/Windows/xp_sp2.py", line 261, in handles
    for h in self._make_handle_array(offset, table_levels):
  File "/Users/gleeda/Work/TESTING/vol_1.4/plugins/overlays/Windows/xp_sp2.py", line 228, in _make_handle_array
    for h in self._make_handle_array(offset, level - 1):
  File "/Users/gleeda/Work/TESTING/vol_1.4/plugins/overlays/Windows/xp_sp2.py", line 237, in _make_handle_array
    if item.Type.Name:
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/obj.py", line 771, in __getattribute__
    return self.m(attr)
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/obj.py", line 756, in m
    parent = self, name = attr)
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/obj.py", line 554, in __init__
    parent = parent, profile = profile)
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/obj.py", line 490, in __init__
    BaseObject.__init__(self, theType, offset, vm, parent = parent, name = name)
  File "/Users/gleeda/Work/TESTING/vol_1.4/volatility/obj.py", line 298, in __init__
    raise InvalidOffsetError("Invalid Address 0x{0:08X}, instantiating {1}".format(offset, name))
volatility.obj.InvalidOffsetError: Invalid Address 0x80800000, instantiating 
Type

Original comment by jamie.l...@gmail.com on 2 Sep 2010 at 5:13

GoogleCodeExporter commented 8 years ago 
Hmmmm, I've updated the summary to give a better idea of the problem.  The 
patch evidently still doesn't do the same thing that 1.3.2 does, so it'll need 
a bit more looking into.  Thanks for the additional information, hopefully 
we'll get this sorted soon...  5:)

Original comment by mike.auty@gmail.com on 2 Sep 2010 at 5:18

GoogleCodeExporter commented 8 years ago 
just to clarify: you get the same crash with or without "--profile=WinXPSP3" 
someone asked me about that ;-)

Also with SP2  I get too many repeated files.  For example, when running 
against the sample_nist0625_xpsp2.mem sample the following output is repeated 
50 times for pid 4:

File   \Documents and Settings\LocalService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat.LOG
File   \Documents and Settings\LocalService\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat
File   \Documents and Settings\Sarah\NTUSER.DAT
File   \Documents and Settings\Sarah\ntuser.dat.LOG
File   \Documents and Settings\Sarah\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat
File   \Documents and Settings\Sarah\Local Settings\Application 
Data\Microsoft\Windows\UsrClass.dat.LOG
File   \Topology     
File   \     
File   \WINDOWS\system32\MsDtc\Trace\dtctrace.log
File   \255     
File   \113

Original comment by jamie.l...@gmail.com on 2 Sep 2010 at 6:27

GoogleCodeExporter commented 8 years ago 
Ok, well, I figured out the difference between the two versions, now we just 
have to figure out which one's right...

In volatility-1.3.2, the table arrays (rather than the ones that hold objects) 
are traversed from 0 to 0x400 in steps of 4 (ADDR_SIZE), whilst the end arrays 
(the ones that hold objects) are traversed from 0 to 0x200 in steps of 8 
(obj_size of _HANDLE_TABLE_ENTRY).

In volatility-1.4, the table arrays and the table entries are traversed from 0 
to 0x200 in steps of 8 (_HANDLE_TABLE_ENTRY size).  Volatility-1.4 also stop 
traversing a table at the first bad offset (which may not have any impact on 
the matter, but does differ from how 1.3 works).

What this is means is that either 1.4 was skipping over half the entries, or 
1.3 was finding additional (incorrect) entries, and returning them.  The 
question is which.  Anyone know enough about this to hazard an answer?  I'm 
happy to produce a patch either way, but at the moment I don't know the exact 
rules for how these tables are supposed to work.  The XP_SP3 generated type for 
HANDLE_TABLE_ENTRY also doesn't really give any clues as to what goes where.

Original comment by mike.auty@gmail.com on 2 Sep 2010 at 7:35

GoogleCodeExporter commented 8 years ago 
Ok, here's a patch in case 1.3 turns out to be the right solution (although, we 
do still need someone to verify this for us).  Please test it and let me know 
if it gives the same output as 1.3.2 at least.

It seems a little odd, because the _HANDLE_TABLE_ENTRY's Object (which is a 
Pointer to Void in both sp2 and sp3) won't actually point to a valid 
_OBJECT_HEADER, unless it's anded with 0x00000007.  So all in all, it might 
just as well be treated as an unsigned int/long.  It's therefore entirely 
unclear what the point of a _HANDLE_TABLE_ENTRY is?

In that patch, it might also be worth using entry.Value rather than 
entry.Object, but only Object is defined for SP2 (in SP3, Value and several 
other members are defined).  This relates to the need for XPSP2 to be 
autogenerated, so we get all the random little members that weren't entered by 
hand.

Anyway, for now, this bug is still waiting on someone to say authoritatively 
exactly how the ObjectTables work.

Original comment by mike.auty@gmail.com on 2 Sep 2010 at 9:04

Attachments:

GoogleCodeExporter commented 8 years ago 
This seems to have fixed the problem.  I've tested on different images SP3 and 
SP2 and don't see this problem now :-)

Original comment by jamie.l...@gmail.com on 2 Sep 2010 at 9:30

GoogleCodeExporter commented 8 years ago 
I'm reasonably confident that Volatility 1.3.2's is correct. At one point we 
had the numbers wrong (we were going off of Schreiber's description, which was 
valid for Win2k and not XP), but there should be a changelog entry for the rev 
where we fixed that. When I put in that fix I also created a test program that 
created a large number of uniquely named mutexes, and verified that handles 
found them all.

Original comment by moo...@gmail.com on 3 Sep 2010 at 7:05

GoogleCodeExporter commented 8 years ago 
Ok, cool, it might be worth running the test program again, if you've still got 
it somewhere?

Since we seem to now have parity with the 1.3.2 version I'm going to mark this 
as Fixed.  Feel free to reopen it if anyone spots something that doesn't look 
right/is missing...  5:)

Original comment by mike.auty@gmail.com on 3 Sep 2010 at 10:25