not parsing large binary registry values correctly

[GoogleCodeExporter]

I originally thought this problem was related to issue 147, but it appears that 
we are incorrectly handling large key values.  To confirm this, I wrote a 
plugin that would compare the binary registry values from those that are in 
memory to registry files taken from the disk from the same snapshot.  Here are 
the details for a Windows 7 SYSTEM registry:

For the SYSTEM registry of machine for a Windows 7 snapshot (win7.vmem):

* 9057 binary values were identical to the registry file from disk.
* 749 Values were different from the disk registry, 739 of which are Volatile 
keys (which are not found in the registry on disk).

Of the 10 non-volatile keys that were different, the following contained 
corrupt data (raw registry data). These are all keys with large data. Format is 
"key => value":

* ControlSet001\Control\ProductOptions => ProductPolicy
* ControlSet002\Control\ProductOptions => ProductPolicy
* ControlSet001\Control\Session Manager\AppCompatCache => AppCompatCache
* ControlSet001\services\rdyboost\Parameters => BootPlan
* ControlSet002\Control\Session Manager\AppCompatCache => AppCompatCache
* ControlSet002\services\rdyboost\Parameters => BootPlan

Others are just different, for valid reasons. The data was valid, just slightly 
different:

* RNG => Seed
* 
ControlSet001\Enum\PCI\VEN_15AD&DEV_0405&SUBSYS_040515AD&REV_00\3&18d45aa6&0&78\
Device Parameters => VidPnLkgTopology
* 
ControlSet001\services\Tcpip\Parameters\Interfaces\{DC6788D3-64B0-47CA-A80E-B1B4
0A7AE656} => DhcpInterfaceOptions
* ControlSet001\Hardware 
Profiles\UnitedVideo\CONTROL\VIDEO\{EB008923-FCC2-4CF3-8ED4-F2C3F70AFFD7}\0000 
=> DefaultSettings.DriverExtra

I was able to confirm the same findings with various other Windows 7 images 
that I have, some from actual cases by using printkey to print out these keys 
that contain large values.  Some of these same large keys are OK on Vista 
image, *but* the values are much smaller than those in Windows 7.  I'm not sure 
what is the border of sizes that are parsed with no issue yet.

So I'm not sure if we want to close out issue 147.  I'm leaving it open because 
there seemed to be some question about it.  I have talked to AW and MHL and we 
seem in agreement that this is more about parsing large registry values than 
pages in transition, so I am opening this issue for discussion.

Original issue reported on code.google.com by jamie.l...@gmail.com on 15 Nov 2011 at 4:46

[GoogleCodeExporter]

Ok, one note: it seems that we do not handle bigdata items, which have a 
signature of "db", at least I don't see it handled in the code...  You can see 
this type on page 4: 
http://sentinelchicken.com/data/TheWindowsNTRegistryFileFormat.pdf

You can see the "db" signature in the raw registry data output in my comment 
from issue 147: http://code.google.com/p/volatility/issues/detail?id=147#c8

Original comment by jamie.l...@gmail.com on 21 Nov 2011 at 9:47

[GoogleCodeExporter]

so it appears that it was implemented in Moyix's registry stuff 
(forensics/win32/rawreg.py +166), but left out of of 2.0 for some reason...

Original comment by jamie.l...@gmail.com on 21 Nov 2011 at 10:46

[GoogleCodeExporter]

Hmmmm, that was probably me that did the conversion, but it's quite a long time 
ago now.  I can take a look back, but at some point the registry code could do 
with a cleanup and bit of a refactor.  If someone could point me at the 
original moyix code (forensics/win32/rawreg.py doesn't seem to be in our tree), 
I'll see what happened in the conversion and try and produce a quick patch, or 
a reason I can't/didn't...  5:)

I can't say how long it'll take, but I will hopefully start having more time in 
December, and certainly towards the end.  5:)

Original comment by mike.auty@gmail.com on 21 Nov 2011 at 11:46

[GoogleCodeExporter]

The original code (which includes BIG_DATA handling) is here:
http://www.cc.gatech.edu/~brendan/volatility/dl/volreg-0.6.tar.gz

Original comment by moo...@gmail.com on 21 Nov 2011 at 11:48

[GoogleCodeExporter]

I've attached an ugly patch for this.  Now we can see that it *was* a big value 
issue since now we can see the correct data for BootPlan:

$ ./vol.py printkey -K "ControlSet001\Services\rdyboost\Parameters" 
--profile=Win7SP0x86 win7.dmp
Volatile Systems Volatility Framework 2.1_alpha
Legend: (S) = Stable   (V) = Volatile

----------------------------
Registry: \REGISTRY\MACHINE\SYSTEM
Key name: Parameters (S)
Last updated: 2011-10-04 17:55:35 

Subkeys:

Values:
REG_BINARY    ReadyBootVolumeUniqueId : (S) 
0x00000000  73 9e 5d 1b 00 00 10 00 00 00 00 00               s.].........
REG_DWORD     ReadyBootPlanAge : (S) 0
REG_BINARY    BootPlan        : (S) 
0x00000000  02 00 0c 00 92 28 00 00 80 18 00 00 9a 00 00 00   .....(..........
0x00000010  04 37 47 4a 9d 82 cc 01 00 00 00 00 00 00 00 00   .7GJ............
0x00000020  00 90 00 00 01 00 00 00 00 90 00 00 00 00 00 00   ................
0x00000030  00 a0 00 00 01 00 00 00 00 30 01 00 00 00 00 00   .........0......
0x00000040  00 00 01 00 01 00 00 00 00 30 02 00 00 00 00 00   .........0......
0x00000050  00 20 00 00 01 00 00 00 00 70 02 00 00 00 00 00   .........p......
0x00000060  00 10 00 00 01 00 00 00 00 90 02 00 00 00 00 00   ................
0x00000070  00 40 00 00 01 00 00 00 00 10 40 00 00 00 00 00   .@........@.....
0x00000080  00 50 00 00 01 00 00 00 00 80 40 00 00 00 00 00   .P........@.....
0x00000090  00 00 01 00 01 00 00 00 00 90 41 00 00 00 00 00   ..........A.....
[snip]
0x00041a80  00 30 00 00 01 00 00 00 00 70 e1 54 01 00 00 00   .0.......p.T....
0x00041a90  00 10 00 00 01 00 00 00 00 80 01 c3 01 00 00 00   ................
0x00041aa0  00 10 00 00 01 00 00 00 00 30 e9 2e 02 00 00 00   .........0......
0x00041ab0  00 10 00 00 01 00 00 00 00 30 f3 2e 02 00 00 00   .........0......
0x00041ac0  00 80 02 00 01 00 00 00 00 40 dd 32 02 00 00 00   .........@.2....
0x00041ad0  00 20 00 00 01 00 00 00 73 9e 5d 1b 00 00 10 00   ........s.].....
0x00041ae0  00 00 00 00                                       ....
REG_SZ        LastBootPlanUserTime : (S) \u200eTue\u200e, \u200eOct \u200e04 
\u200e11, 01:55:35 
PM\uc4cc\u70d3\u2892\u40e4\u1000\uf198\u7601\ud704\ud158\u82be\u01cc\u3704\u4a47
\u829d\u01cc\u1ae4\u1878\xea

Original comment by jamie.l...@gmail.com on 21 Nov 2011 at 11:53

Attachments:

• rawreg_diff.patch

[GoogleCodeExporter]

Thanks, both of you, I'll try and get sometime to review this in the next few 
days.  Feel free to poke me (indirectly on here, or directly on IRC) if I 
haven't muttered much in a week or so...  5;)

Original comment by mike.auty@gmail.com on 21 Nov 2011 at 11:59

[GoogleCodeExporter]

thanks moyix, I forgot to include the link :-(

thanks ikelos.  I'll see about cleaning up the patch if you don't have time.  
at least in the meantime you have something to look at :-)

Original comment by jamie.l...@gmail.com on 21 Nov 2011 at 11:59

[GoogleCodeExporter]

Ok, I've cleaned up the patch a bit.  If no one objects by Friday it should be 
committed.  Please let me know if you have any thoughts/objections/comments..

Original comment by jamie.l...@gmail.com on 30 Nov 2011 at 5:41

Attachments:

• rawreg_diff2.patch

[GoogleCodeExporter]

sorry, this should be the correct patch.

Original comment by jamie.l...@gmail.com on 30 Nov 2011 at 5:45

Attachments:

• rawreg_diff2.patch

[GoogleCodeExporter]

Patch applied fine and my results match with yours for the BootPlan value. 

Original comment by michael.hale@gmail.com on 30 Nov 2011 at 6:57

[GoogleCodeExporter]

Awesome!  Thanks for testing :-D

Original comment by jamie.l...@gmail.com on 30 Nov 2011 at 7:00

[GoogleCodeExporter]

This issue was closed by revision r1156.

Original comment by michael.hale@gmail.com on 13 Dec 2011 at 2:58

• Changed state: Fixed