Extract XOR Keys using Metadata

[malvidin]

Proposed fix for enhancement request #83. This extracts the single byte XOR key as an integer represented as a string, or as a representation of a dict of multiple XOR key locations. Because of the way that YARA parses metadata strings (https://github.com/VirusTotal/yara/issues/1242), some strings cannot be stored in the metadata.

• Added XOR key creation using metadata (issues in YARA #1242)
• Added xor keyword to dispatcher.yar XOR rule
• Removed 254 rules replaced by XOR rule
• Added tests for XOR key extraction

[mlaferrera]

Thanks for this PR @malvidin!

I have a few requests before merging:

• [ ] Rename the xor_pt_ metadata key to something more intuitive
• [ ] Update the yara plugin README with the proper use and description
• [ ] Bump versions in setup.py and yarascan.stoq

[malvidin]

I changed the metadata key to xor_plaintext_for_string_, which is more explicit. I did not conduct a survey on intuitiveness for either name.

The XOR extracted information is not provided for the Worker plugin, but that is easy to implement if desired.

[mlaferrera]

Thanks again, @malvidin !

Is there any reason why when xor_first_match is default/True, that it shouldn't return a tuple of location, string key name, and the XOR byte(s)? It seems somewhat cumbersome to handle them differently when parsing the results. For example, replacing the current logic with:

                key = self._xor_extract_key(match_bytes, xor_pt_bytes)
                if key:
                    xor_info.append((offset, label, key))
                    if self.xor_first_match:
                        break
            if xor_info:
                match['meta']['xor_info'] = repr(xor_info)

[malvidin]

I don't have visibility into how the xor module works, so I provided a default that would replicate current functionality.

[mlaferrera]

Thanks, and good point. I'm going to make an internal note that we will probably want to deprecate the single byte string in favor of a list.

Thanks again for all of the help and work on this!