Closed olii closed 3 years ago
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
:memo: Please visit https://cla.developers.google.com/ to sign.
Once you've signed (or fixed any issues), please reply here with @googlebot I signed it!
and we'll verify it.
ℹ️ Googlers: Go here for more info.
Great contribution, thanks!
@googlebot I signed it!
This PR adds code that releases GIL while compiling Yara rules. We noticed that when we compile rules in parallel there is no speedup in comparison to sequential compilation.
The bottleneck is that GIL is not released in a function that is CPU intensive. I added the code that releases and reacquires GIL. This eliminates the bottleneck in the multithreaded environment.
The second commit fixes the GIL locking in the callback. The callback is called in case the Yara rule is invalid.
Here is the code I used for benchmarks.