Yarascan should use a DiscontigScanner (esp. bad speed on x64)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Run the following commands on a 64 bit image:

# Scan in kernel space.
time python vol.py -f ~/images/win7_trial_64bit.raw --profile=Win7SP0x64 
--debug yarascan --kernel -Y "DumpIt"

2.
# Scan in process space.
time python vol.py -f ~/images/win7_trial_64bit.raw --profile=Win7SP0x64 
--debug yarascan -Y "DumpIt"

What is the expected output? What do you see instead?

I expect to see hits for the DumpIt keyword. Instead I get these results:

- For the first search in kernel space it does not complete in a reasonable 
time.

- For the second example, is starts to work, but then the volatility resident 
memory usage hit 5.5 gb on my machine, where it got oom killed.

Please use labels and text to provide additional information.

The first problem is due to the code here:
http://code.google.com/p/volatility/source/browse/trunk/volatility/plugins/malwa
re/malfind.py#331

reading every buffer from kdbg.MmSystemRangeStart.dereference_as("address") (on 
this image this is 0xffff080000000000) until 2**64-1 - this is around 247 
terabytes.

The second problem occurs due to this code:
http://code.google.com/p/volatility/source/browse/trunk/volatility/plugins/malwa
re/malfind.py#343

               for vad, data in task.get_vads():
                    for hit in rules.match(data = data):

Which calls:

        for vad in self.VadRoot.traverse():
...
            data = process_space.zread(vad.Start, vad.End - vad.Start + 1)

i.e. there is no bound on the size of this buffer, and if the vad covers a 
large range, this reads it all in at once.

Original issue reported on code.google.com by scude...@gmail.com on 18 Jul 2012 at 4:53

[GoogleCodeExporter]

Hey Scudette, 

Thanks for looking into it. A while ago, you were going to write me an example 
DiscontigScanner which would solve these issues - is that still possible? 

Original comment by michael.hale@gmail.com on 18 Jul 2012 at 5:03

[GoogleCodeExporter]

Ah, I may work with someone later on DiscontigScanner if you're not around. So 
I looked into the process mode issue and it has nothing to do with yarascan 
actually. Its a different issue affecting all vad parsing in all branches, so 
that will be investigated today too ;-)

Original comment by michael.hale@gmail.com on 18 Jul 2012 at 1:32

[GoogleCodeExporter]

Hi Michael,
  Yes I ran into the issue while writing the DiscontigScanner for yara. I have some code which works now and I can backport to trunk if you like. The code also solves the issue of the vad scanning.

Thanks
Michael.

Original comment by scude...@gmail.com on 18 Jul 2012 at 2:10

[GoogleCodeExporter]

Sure, that would be great ;-)

Original comment by michael.hale@gmail.com on 18 Jul 2012 at 2:25

[GoogleCodeExporter]

Hope you don't mind - I'm going to split this up into two issues, so I can 
paste some debugging output wrt the process vads into a dedicated spot. 

Original comment by michael.hale@gmail.com on 18 Jul 2012 at 5:13

• Changed title: _Yarascan should use a DiscontigScanner (esp. bad speed on x64) _

[GoogleCodeExporter]

This is a patch to fix the discontiguous scanning issue. It also changes the 
prototype of get_vads() to not return the entire vad region in one string (The 
region can be very large).

Original comment by scude...@gmail.com on 19 Jul 2012 at 4:31

Attachments:

• svn.diff

[GoogleCodeExporter]

Thanks man! Going to apply and do some testing and I will report back shortly 
;-)

Original comment by michael.hale@gmail.com on 19 Jul 2012 at 4:39

[GoogleCodeExporter]

This issue was closed by r2077. 

Original comment by michael.hale@gmail.com on 24 Jul 2012 at 3:09

• Changed state: Fixed