Adding Offsets to pslist and general formatting of output of psscan* plugins

[GoogleCodeExporter]

Some of us were discussing adding offsets to the pslist output to allow one to 
dump processes based on offset without having to scan for them to get the 
offset.  There are other valid reasons for adding the offset to pslist in 
addition to better general usability...

I am trying to decide how to format the output in a way that will be consistent 
with the psscan* plugins.  At the moment I have output like the following 
(patch included):

$ python vol.py -f ds_fuzz_hidden_proc.img pslist
Volatile Systems Volatility Framework 1.4_rc1
Name                 Pid    PPid   Thds   Hnds   Physical Offset   Time         

System                    4      0     51    254 0x01bcc830       1970-01-01 
00:00:00 
smss.exe                360      4      3     19 0x019e4670       2008-11-26 
07:38:11 
csrss.exe               596    360     10    322 0x01a1bd78       2008-11-26 
07:38:13 
winlogon.exe            620    360     16    503 0x01a2b100       2008-11-26 
07:38:14 
services.exe            672    620     15    245 0x01a3ba78       2008-11-26 
07:38:15 
lsass.exe               684    620     21    347 0x019dbc30       2008-11-26 
07:38:15 
svchost.exe             844    672     19    198 0x01a59d70       2008-11-26 
07:38:18 

[snip]

and psscan has output like the following:

$ python vol.py -f ds_fuzz_hidden_proc.img psscan
Volatile Systems Volatility Framework 1.4_rc1
PID    PPID   Time created             Time exited              Offset     PDB  
      Remarks
------ ------ ------------------------ ------------------------ ---------- 
---------- ----------------
     0      0                                                   0x00552a20 0x00319000 Idle            
   992    660 2008-11-15 23:43:25                               0x0181b748 0x08140260 alg.exe         
  1372   1064 2008-11-26 07:39:38                               0x01843b28 0x08140180 wuauclt.exe     
   560   1064 2008-11-26 07:44:57                               0x0184e3a8 0x081402a0 wscntfy.exe     
   512    672 2008-11-26 07:38:53                               0x018557e0 0x08140260 alg.exe         
   940   1516 2008-11-26 07:43:39      2008-11-26 07:45:49      0x0185dda0 0x081401a0 cmd.exe         
  1756    672 2008-11-26 07:38:45                               0x018a13c0 0x08140220 VMwareService.e 
   808    620 2008-11-26 07:45:22      2008-11-26 07:45:40      0x018af020 0x08140280 taskmgr.exe   

[snip]

There doesn't seem to be an existing rule about where offsets should be placed. 
 MHL says he prefers offsets to be on the far left.  I'm putting this issue 
here to open it up to a vote on how we'd like to format this.

Original issue reported on code.google.com by jamie.l...@gmail.com on 7 Mar 2011 at 7:10

[GoogleCodeExporter]

Sorry meant to patch from the root directory instead...

Original comment by jamie.l...@gmail.com on 7 Mar 2011 at 7:11

Attachments:

• pslist_offset.patch

[GoogleCodeExporter]

sorry... left off the header in the patch, not that that is important, but here 
it is anyway.  It's definitely Monday...

Original comment by jamie.l...@gmail.com on 7 Mar 2011 at 7:16

Attachments:

• pslist_offset.patch

[GoogleCodeExporter]

Yeah, I prefer the far left for offsets, but I also think we should consider a 
way to make this consistent across all plugins and not just the process listing 
ones. I wasn't overly bothered by this, but now that there's an issue, I may as 
well voice some opinions ;-)

* the process name is in the "Name" column for pslist but in the "Remarks" 
column for psscan. is there a reason the column isn't "Name" for psscan as well?

* some plugins write dashes under the column name. for consistency, should we 
pick to always or never write dashes?

* some scanners like sockscan/connscan don't show offsets. there are some 
memory forensics tools though i won't mention the name (you all know anyway) 
that display obvious duplicates of many objects. by displaying the offsets 
instead of just the fields, we can prevent it from appearing as if Volatility 
also shows duplicates. 

Original comment by michael.hale@gmail.com on 8 Mar 2011 at 3:00

[GoogleCodeExporter]

I prefer far left for offsets if there's only one.  It can get a bit much if 
there's two, or even more.  I'd keep the far left for the primary offset (one 
that'll be used as a parameter for other plugins).

I think this may stray into issue 11 territory, whereby all plugins that 
produce tabular output can do so more easily by defining a UI object and 
populating that.  The UI can then decide whether there's bars, or whether they 
should be output at HTML, or put into an SQL table or whatever.  I don't think 
any of this should really *block* 1.4, although making the specific corrections 
mentioned should probably be done before the release...

The patch looks good, I'm just wondering if having the physical offset is more 
useful than having the virtual offset?  I thought most of our plugins took 
virtual offset parameters?  Are there any plugin where you'd provide the 
process's offset (I honestly don't remember)?

Original comment by mike.auty@gmail.com on 8 Mar 2011 at 9:36

[GoogleCodeExporter]

Actually, the usual methodology is to provide a physical offset taken from the 
psscan plugins.  All the "dumping" plugins have an option for a physical 
offset.  That is why I have provided a physical offset here, just to keep with 
that methodology.  Otherwise we could make the "dumping" plugins take in a 
virtual address in addition to the physical one... but I'm not sure how 
confusing it will be to our general users to have two offsets like that...  

MHL and I were talking about this earlier, he suggested something like:

1) scanners show physical 
2) others show virtual by default 
3) others can be displayed in physical with command line switch

Original comment by jamie.l...@gmail.com on 8 Mar 2011 at 9:47

[GoogleCodeExporter]

Ok, sounds like a plan to me.  5:)

Original comment by mike.auty@gmail.com on 8 Mar 2011 at 9:51

[GoogleCodeExporter]

OK, how about this for pslist/psscan*:

Virtual offset of pslist:

$ python vol.py -f win7.dd --profile=Win7SP0x86 pslist
Volatile Systems Volatility Framework 1.4_rc1
 Offset     Name                 PID    PPID   Thds   Hnds   Time 
---------- -------------------- ------ ------ ------ ------ ------------------- 
0x84133a30 System                    4      0     88    486 2010-06-16 15:24:58 

0x852e7020 smss.exe                252      4      2     29 2010-06-16 15:24:58 

0x859f3d40 csrss.exe               352    316      9    406 2010-06-16 15:25:12 

0x85a5a530 wininit.exe             392    316      3     75 2010-06-16 15:25:15 

0x85a5f530 csrss.exe               400    384     10    361 2010-06-16 15:25:15 

0x859f5bc0 winlogon.exe            464    384      3    112 2010-06-16 15:25:18 

0x85b0b318 services.exe            508    392      6    185 2010-06-16 15:25:18 

0x85d393f8 lsass.exe               516    392      6    584 2010-06-16 15:25:18 

0x841d1750 lsm.exe                 524    392     10    143 2010-06-16 15:25:18 

0x85d5b8f8 svchost.exe             628    508      9    361 2010-06-16 15:25:19 

[snip]

Physical offset (notice -P flag) of pslist:

$ python vol.py -f win7.dd --profile=Win7SP0x86 pslist -P
Volatile Systems Volatility Framework 1.4_rc1
 Offset     Name                 PID    PPID   Thds   Hnds   Time 
---------- -------------------- ------ ------ ------ ------ ------------------- 
0x3fff3a30 System                    4      0     88    486 2010-06-16 15:24:58 

0x3ece7020 smss.exe                252      4      2     29 2010-06-16 15:24:58 

0x3e7f3d40 csrss.exe               352    316      9    406 2010-06-16 15:25:12 

0x3e45a530 wininit.exe             392    316      3     75 2010-06-16 15:25:15 

0x3e45f530 csrss.exe               400    384     10    361 2010-06-16 15:25:15 

0x3e7f5bc0 winlogon.exe            464    384      3    112 2010-06-16 15:25:18 

0x3e50b318 services.exe            508    392      6    185 2010-06-16 15:25:18 

0x3e3393f8 lsass.exe               516    392      6    584 2010-06-16 15:25:18 

0x3ff11750 lsm.exe                 524    392     10    143 2010-06-16 15:25:18 

0x3e35b8f8 svchost.exe             628    508      9    361 2010-06-16 15:25:19 

[snip]

New output format for psscan*:

$ python vol.py -f win7.dd --profile=Win7SP0x86 psscan3 --no-cache
Volatile Systems Volatility Framework 1.4_rc1
 Offset     Name             PID    PPID   PDB        Time created             Time exited    
---------- ---------------- ------ ------ ---------- ------------------------ 
------------------------ 
0x3e025ba8 svchost.exe        1116    508 0x3ecf1220 2010-06-16 15:25:25    
0x3e04f070 svchost.exe        1152    508 0x3ecf1340 2010-06-16 15:27:40    
0x3e144c08 dwm.exe            1540    832 0x3ecf12e0 2010-06-16 15:26:58    
0x3e145c18 TPAutoConnSvc.     1900    508 0x3ecf1360 2010-06-16 15:25:41    
0x3e3393f8 lsass.exe           516    392 0x3ecf10e0 2010-06-16 15:25:18    
0x3e35b8f8 svchost.exe         628    508 0x3ecf1120 2010-06-16 15:25:19    
0x3e383770 svchost.exe         832    508 0x3ecf11a0 2010-06-16 15:25:20    
0x3e3949d0 svchost.exe         740    508 0x3ecf1160 2010-06-16 15:25:20    
0x3e3a5100 svchost.exe         872    508 0x3ecf11c0 2010-06-16 15:25:20    
[snip]

Original comment by jamie.l...@gmail.com on 28 Mar 2011 at 3:54

Attachments:

• pslist_psscan.patch

[GoogleCodeExporter]

Looks great!  5:)

A couple comments on the patch:

* The PHYSICAL_OFFSET flag won't actually affect the cached results, so you 
should probably include "cache_invalidator = False" into the add_option 
parameter list, otherwise changing between -P and not would require it to 
reread the entire disk.

* Rather than reloading the AS as kernel_address_space, it would probably be 
easier to use the vm task was made with task.obj_vm.vtop(task.obj_offset).

* Just a minor point now, since it's going to get stuck in a big "{}".format 
function anyway, I might not bother with the "0x{0:08x}".format calls, and just 
change the "{0:10}" to "{0:#010x}".  5;)

Otherwise looks good to me, and I'm happy to commit it if you'd like (I can 
make the above changes in the process).  Just lemme know...  5:)

Original comment by mike.auty@gmail.com on 28 Mar 2011 at 8:53

[GoogleCodeExporter]

Cool :-)  all of that makes sense.  Feel free to fix as you like and commit.  I 
will send patches for the other items for which we want offsets.  Thanks 
ikelos! :-)

Original comment by jamie.l...@gmail.com on 28 Mar 2011 at 12:34

[GoogleCodeExporter]

Ok, commited that as r904.  I also made the -P flag change the Offset header, 
so people know which type of offset volatility thinks it's showing...  5:)  
Looking forward to the next patch!  5:)

Original comment by mike.auty@gmail.com on 28 Mar 2011 at 6:27

[GoogleCodeExporter]

Thanks ikelos!  Here is a patch for connections/connscan2 and sockets/sockscan. 
 Feel free to change as needed and commit.

Original comment by jamie.l...@gmail.com on 28 Mar 2011 at 11:24

Attachments:

• connections_sockets.patch

[GoogleCodeExporter]

This issue was closed by revision r907.

Original comment by mike.auty@gmail.com on 28 Mar 2011 at 11:37

• Changed state: Fixed

[GoogleCodeExporter]

Thanks again, ikelos!  Here is another patch for regkeys and files :-)

Original comment by jamie.l...@gmail.com on 29 Mar 2011 at 3:03

Attachments:

• files_regkeys.patch