Process Memory Scanning Issue on Windows 10

[Neo23x0]

We noticed a problem with the process memory scanning on Windows 10 when using the 64bit version of YARA. (also counts for Python and Golang bindings)

To reproduce:

YARA 3.7.x process memory scan on Windows 10 64bit

32bit yara – 1 Rule 0.03 sec 32bit yara – 100 Rules 0.03 sec

64bit yara – 1 Rule 0.5 sec 64bit yara – 100 Rules 12 sec <<<

The number of rules involved in the process memory scan increases the scan duration proportionally but it shouldn't. Scanning a process memory on Windows 10 with 500 rules takes a significant amount of time. (~250 seconds)

We've noticed this issue using the golang bindings but you can reproduce it with the public yara binaries as well.

Details Using yara32 on Windows 10

C:\yara>yara.exe 1068
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-0.yar rules\rule-0.yac]
[yara37_32 rules\rule-0.yac 1068]
took: 0.03387 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-1.yar rules\rule-1.yac]
[yara37_32 rules\rule-1.yac 1068]
took: 0.0346062 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-10.yar rules\rule-10.yac]
[yara37_32 rules\rule-10.yac 1068]
took: 0.0329233 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-11.yar rules\rule-11.yac]
[yara37_32 rules\rule-11.yac 1068]
took: 0.034042 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-12.yar rules\rule-12.yac]
[yara37_32 rules\rule-12.yac 1068]
took: 0.0327866 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-13.yar rules\rule-13.yac]
[yara37_32 rules\rule-13.yac 1068]
took: 0.0320835 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-2.yar rules\rule-2.yac]
[yara37_32 rules\rule-2.yac 1068]
took: 0.0329113 seconds
[yarac37_32 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-3.yar rules\rule-3.yac]
[yara37_32 rules\rule-3.yac 1068]
took: 0.0316628 seconds

Using yara64 on Windows 10

C:\yara>yara.exe 1068
[yarac37 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-0.yar rules\rule-0.yac]
[yara37 rules\rule-0.yac 1068]
took: 10.7976936 seconds
[yarac37 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-1.yar rules\rule-1.yac]
[yara37 rules\rule-1.yac 1068]
took: 11.8049727 seconds
[yarac37 -d filename=xxx -d extension=xxx -d filetype=xxx -d filepath=xxx rules\rule-10.yar rules\rule-10.yac]
[yara37 rules\rule-10.yac 1068]
took: 10.989832700000001 seconds

To reproduce this, you could use my signature set https://github.com/Neo23x0/signature-base/blob/master/yara/apt_eqgrp_apr17.yar

[Neo23x0]

@hillu who investigated that issue came to the conclusion that this is most likely caused by the high number of magic header checks in my rules, which are often in the form of uint16(0) == 0x5a4d.