overlay readability changes

[GoogleCodeExporter]

As suggested by Scudette, the following overlay syntax is very hard to read:

win2k3sp0x86overlays['_EPROCESS'][1]['VadRoot'][1] = ['_MM_AVL_TABLE']
win2k3sp0x86overlays['VOLATILITY_MAGIC'][1]['DTBSignature'][1] = 
['VolatilityMagic', dict(value = "\x03\x00\x1B\x00")]
win2k3sp0x86overlays['VOLATILITY_MAGIC'][1]['KPCR'][1] = ['VolatilityKPCR', 
dict(value = 0xffdff000, configname = 'KPCR')]

 We should just include these into the overlay directly or do an update
(either through the update or the add_overlay() method).

win2k3sp0x86overlays.update({
   "_EPROCESS": [ None, {
        "VadRoot": [ None, ['_MM_AVL_TABLE']]
 }]
})

Original issue reported on code.google.com by michael.hale@gmail.com on 10 Jan 2012 at 2:40

[GoogleCodeExporter]

Update would wipe out all the other overlaid members of _EPROCESS besides 
VadRoot.

Apply_overlay should be called merge_dictionaries, and is not specific to a 
profile.  My current thinking is to pull this function out of profile and put 
it either into utils.py, or a profile.py (and transfer Obj.Profile over to 
profile.py as well).

Obj would need to be left with a Profile for compatibility reasons, and 
apply_overlay would call merge_dictionaries after issuing a deprecation warning.

That's the plan, but it's not urgent since the overlays have already been 
refactored, and most of the redundant entries have been removed.  If people see 
any problems with, or have any concerns, this is the place to raise them.  I 
won't make any changes without a few OKs from other devs...

Original comment by mike.auty@gmail.com on 22 Jan 2012 at 7:12

[GoogleCodeExporter]

There are a couple of issues here. First we do need to work towards a
situation where we do not "steal" overlays from other profiles and
just change them a bit - each profile should be completely self
contained - even if this means duplicating code.

Despite this it is still possible to call the apply_overlay method
multiple times and it will do the correct merging for you:

http://code.google.com/p/volatility/source/browse/branches/scudette/volatility/o
bj.py#1095

http://code.google.com/p/volatility/source/browse/branches/scudette/volatility/p
lugins/overlays/windows/win7_sp1_x64.py#120

Not unless we start calling the overlays dictionaries - that name is
too generic. Apply_overlays does exactly that - it applies the overlay
(it will not accept arbitrary dictionaries - only valid overlay dicts
which has a certain structure.).

So I tried initially to factor profiles out of obj.py and discovered
that it really didnt make any sense - the problem is that profiles are
essentially object factories - i moved the Object() factory into the
profile and am currently working on breaking the interdependency
between the profile and the address space.

Ultimately, the profile has a single factory method (Object() - I kept
the name) and that is used to create any object instance by name. The
profile modules just build one of these by calling apply_overlay() and
loading their vtypes. Now that you can call apply_overlay repeatably,
it makes adding the overlays much more readable since we can have them
split into functionality groups.

Michael.

Original comment by scude...@gmail.com on 23 Jan 2012 at 12:26

[GoogleCodeExporter]

This should be improved with the new profile system.  Overlays will be applied 
as part of modifications, each of which will be a logical unit for a set of 
profiles and make things much more readable.

Original comment by mike.auty@gmail.com on 12 Feb 2012 at 8:44

[GoogleCodeExporter]

Original comment by mike.auty@gmail.com on 12 Feb 2012 at 10:05

• Added labels: Milestone-2.1.x

[GoogleCodeExporter]

MHL, are you happy for this issue to be closed?

Original comment by mike.auty@gmail.com on 21 Feb 2012 at 9:39

[GoogleCodeExporter]

Original comment by michael.hale@gmail.com on 21 Feb 2012 at 10:13

• Changed state: Fixed