printing addresses consistently for x86 & x64

[GoogleCodeExporter]

Hey guys, 

I noticed something strange with kdbgscan. Not sure what the issue is yet. 

Potential KDBG structure addresses (P = Physical, V = Virtual):
 _KDBG: V 0xf80002837070  (Win7SP1x64)
 _KDBG: P 0x02837070  (Win7SP1x64)
 _KDBG: V 0xf80002837070  (Win7SP0x64)
 _KDBG: P 0x02837070  (Win7SP0x64)

So it says virtual address is 0xf80002837070, but in windbg that address is 
invalid - and I think it should be  fffff800`02837070. 

This got me thinking...all of our plugins that print virtual addresses have 
fixed width column headers that get all throw off when we squeeze a 64-bit 
address in there. Can we come up with a way to adjust the column headers 
depending on the size of the address being printed (or just always print 64-bit 
width even for 32-bit addresses - just make the high order 32-bit all 0's)?

Original issue reported on code.google.com by michael.hale@gmail.com on 24 Jan 2012 at 6:34

[GoogleCodeExporter]

Original comment by mike.auty@gmail.com on 21 Feb 2012 at 9:27

• Added labels: Milestone-2.1.x

[GoogleCodeExporter]

Hiya, so bits 48 upwards of a virtual address don't actually matter, and I 
believe are supposed to be 0.  Section 4.5 of the Intel arch specs don't say a 
lot about the higher bits, while the AMD specs, in section 5.3.3, say that bits 
63-48 should be a sign extension of bit 47 (so basically repeats of bit 47).  
Volatility currently doesn't differentiate, so vtop(0xfffff80002837070) == 
vtop(0x1234f80002837070) == vtop(0xf80002837070).  

One difficulty with getting the column headers the correct size is that the 
render_text function usually doesn't have access to the profile, so at the time 
we want to write the titles we don't know how wide they should be.  We can't 
use the objects in the first row because there may not be a first row, or there 
may be NoneObjects.  We could add code to the Command class so that calculate 
can set the appropriate column widths, and the command hangs onto them for use 
in calculate but it'd only be useful for tabular output.  Does it seem 
reasonable to have such specific code for that task?

Original comment by mike.auty@gmail.com on 11 Mar 2012 at 7:48

[GoogleCodeExporter]

Ah that's interesting about the 48 bits Mike, I didn't know that - thanks for 
looking into it. 

So the 0xfffff80002837070 vs 0xf80002837070 thing is actually related to 184 - 
please see that issue for a description.

Original comment by michael.hale@gmail.com on 9 Apr 2012 at 3:47

[GoogleCodeExporter]

Hmmmm, so is that the kpcrscan's checks don't match because they need masking 
to 48-bits?  Would that be flexible in the future, or should we be adding a 
VolMagic value in the profiles for it?  I'd rather not do that...

I'm still working on this, I just need poking occasionally.  5;)  I'll see if I 
can get it a bit further by tomorrow...

Original comment by mike.auty@gmail.com on 9 Apr 2012 at 3:52

[GoogleCodeExporter]

So regarding the output formatting, it doesn't look like it should be that big 
a deal.  I've attached a patch for pslist for an example.  Output: 

$ ./vol.py -f memory.dd --profile=Win2003SP2x64 pslist
Volatile Systems Volatility Framework 2.1_alpha
 Offset(V)          Name                 PID    PPID   Thds   Hnds   Time 
------------------ -------------------- ------ ------ ------ ------ 
------------------- 
0xfffffadfa2e9bc20 System                    4      0     83  12803 1970-01-01 
00:00:00       
0xfffffadfa20d5a80 smss.exe                432      4      2     21 2012-03-01 
08:09:42       
0xfffffadfa1384c20 csrss.exe               492    432     14    913 2012-03-01 
08:09:45       
0xfffffadfa2146590 winlogon.exe            516    432     20    643 2012-03-01 
08:09:45       
0xfffffadfa24c5c20 services.exe            564    516     18    421 2012-03-01 
08:09:46       
0xfffffadfa1353c20 lsass.exe               576    516     40    797 2012-03-01 
08:09:46          

[snip]    

BTW, I think we should also differentiate between x64 and x86 processes on a 
x64 system.  I'll probably have to open a separate issue for this, since 
there's a lot of other things we'll need to do regarding x86 processes on a x64 
system regarding Dlls... but until then I've provided another patch for pslist 
here to show x86 processes.  Output here shows x86 processes denoted with 
"**32" like tasklist:

$ ./vol.py -f memory.dd --profile=Win2003SP2x64 pslist
Volatile Systems Volatility Framework 2.1_alpha
 Offset(V)          Name                 PID    PPID   Thds   Hnds   Time 
------------------ -------------------- ------ ------ ------ ------ 
------------------- 
0xfffffadfa2e9bc20 System                    4      0     83  12803 1970-01-01 
00:00:00       
0xfffffadfa20d5a80 smss.exe                432      4      2     21 2012-03-01 
08:09:42        

[snip]

0xfffffadfa13c0590 FrameworkServic **32   1480    564     31    493 2012-03-01 
08:09:48      
0xfffffadfa11efc20 mfevtps.exe            1672    564      4    160 2012-03-01 
08:09:49       
0xfffffadfa0e7ac20 mfeann.exe **32        1676   1628     11    209 2012-03-01 
08:09:49    

[snip]

Original comment by jamie.l...@gmail.com on 24 Apr 2012 at 4:53

Attachments:

• pslist_address.patch
• pslist_address_and_x86_proc.patch

[GoogleCodeExporter]

hrmmm ok if we are wanting to avoid if/else for all plugins that requires more 
though.  Somehow I missed that that's what we wanted to do... 

Original comment by jamie.l...@gmail.com on 24 Apr 2012 at 5:24

[GoogleCodeExporter]

Ok, so the plan was (when I had time) to get a new function on commands.Command 
objects that takes in a series of titles, types and the config object.  The 
idea then is to instantiate a dummy AS, pull out the size of pointers (and 
potentially with a bit of cleverness any other types that are handed in) and 
the appropriately pad/align the titles, and return a size array for each of the 
rows to format to.

At that point, there's two choices: a) hand back the sizes for each row or b) 
hang on to the formatting values in the command, and provide a "row" function 
that outputs a row appropriately.  Unfortunately, this isn't trivial, but it 
might be worth working towards that goal and throw out the code for others to 
hack incremental improvements onto.  At some point, I'll try and mock something 
up, but it won't be for a little bit (and really, we ought to be testing 2.1, 
I'm not sure this is a solid blocker, it might be worth rolling a 2.1.1 shortly 
after)...

Original comment by mike.auty@gmail.com on 24 Apr 2012 at 6:53

[GoogleCodeExporter]

Hey Mike, 

Sounds like a plan. While this issue is definitely different in nature than, 
say Issue #194, I think its still important to try and tackle before a release, 
just for cleanliness/organization's sake. Some plugins look okay on x64, for 
example pslist, since it just has the single x64 address on the left column. 
But others, such as vadwalk are almost impossible to read:

Volatile Systems Volatility Framework 2.1_alpha
************************************************************************
Pid:      4
Address  Parent   Left     Right    Start    End      Tag
fffffadfe7193f10 00000000 fffffadfe6008140 fffffadfe776e8e0 00050000 0014ffff 
Vad 
fffffadfe6008140 fffffadfe7193f10 fffffadfe7a7e350 00000000 00040000 00040fff 
Vad 
fffffadfe7a7e350 fffffadfe6008140 00000000 00000000 00010000 00033fff Vad 
fffffadfe776e8e0 fffffadfe7193f10 00000000 fffffadfe7aa4220 77ec0000 77ff8fff 
Vad 
fffffadfe7aa4220 fffffadfe776e8e0 00000000 00000000 7ffe0000 7ffeffff Vadl

Original comment by michael.hale@gmail.com on 9 May 2012 at 3:19

[GoogleCodeExporter]

So I implemented the plan in comment 7, and it's now sitting in the issue190 
branch on subversion.  Please could anyone who's interested check this out, try 
out the converted plugins (pslist, dlllist, memmap, vadinfo, vadwalk) and 
please either help covert or provide me a list of plugins that still need 
converting?

Original comment by mike.auty@gmail.com on 19 May 2012 at 2:49

[GoogleCodeExporter]

Hey Mike, 

Looks good, the vadwalk is definitely more readable now ;-)

Here's a list of plugins that print at least one virtual offset or address and 
haven't been converted yet:

connections 
sockets
handles
modscan
modules

Do you want to try and take care of those and I'll do the same for the malware 
plugins? 

Original comment by michael.hale@gmail.com on 21 May 2012 at 3:16

[GoogleCodeExporter]

I just wanted to say, this looks awesome! :-)  

Let me know if you need some help converting some of these other plugins

Original comment by jamie.l...@gmail.com on 21 May 2012 at 4:00

[GoogleCodeExporter]

Ok, turns out it's now pretty quick to convert most plugins (only ones like 
vadinfo are a bit annoying to do, because they interleave text with data).  So 
I've converted over that whole list...

Jamie, feel free to jump in and convert any remaining ones.  Jamie and MHL, do 
gimme a shout if you hit any tricky cases/weird ones.

If everyone's happy with the newly produced output, I'll merge it into trunk 
and we can close off this issue (and finish any conversion in trunk).  Any 
objections or noticable problems?

Original comment by mike.auty@gmail.com on 21 May 2012 at 10:57

[GoogleCodeExporter]

Hey Mike, 

Nothing too tricky yet, I've converted the malware plugins (see patch which 
should apply to r1767 of the issue190 branch). Everything seems to be OK...you 
can probably merge into trunk ;-)

Original comment by michael.hale@gmail.com on 22 May 2012 at 1:59

Attachments:

• issue_190_malware.patch

[GoogleCodeExporter]

Ok, I'm gonna mark this as fixed.  Feel free to add plugins that haven't been 
converted on the end here, and I'll convert them when I can...  5:)

Original comment by mike.auty@gmail.com on 22 May 2012 at 8:29

• Changed state: Fixed