search

airbnb / binaryalert

BinaryAlert: Serverless, Real-time & Retroactive Malware Detection.
https://binaryalert.io
Apache License 2.0
1.41k stars 187 forks source link

unit tests failing with latest yara rules #74

Closed crobo1337 closed 6 years ago

crobo1337 commented 7 years ago

Background

It looks like the latest build of neo23x0's yara ruleset is breaking this build? Based on what i've been able to find it looks like there is possibly some type of version mismatch with the yara-python packages used. Any time a yara rule has a condition that calls pe.imphash the unit tests fail on build_analyzer and compile_rules with [yara.syntaxerror invalid field name"imphash"] I've tried to clone a fresh copy of everything and rebuild from scratch, but I get the same error. I've also tried to pull the latest yara repos down, but no joy there either.

Has anyone successfully implemented newly released yara rules on this build?

austinbyers commented 7 years ago

@crobo1337 problems with *hash can happen if your system doesn't have the OpenSSL development libraries. I just cloned the repo and compiled all the rules with no issue on MacOS. What system are you running on? If it's linux-based, try installing the openssl-devel.x86_64 package before a pip install of the dependencies.

Thanks for flagging! We'll update the documentation once your problem is resolved

crobo1337 commented 7 years ago

cool, ill give it a go and let you know what I find out, thanks.

edit: and to answer your question, i'm running this on amazon linux

austinbyers commented 7 years ago

Yeah, in that case you should be able to yum install openssl-devel.x86_64

BinaryAlert is Python3.6, so you may also need to install that (if you haven't already). The analyzer README lists the full steps we had to take to install yara-python in Amazon linux.

crobo1337 commented 7 years ago

I just nuked the entire ec2 instance i was working on and started fresh, followed the install docs to the letter, with the exception of installing openssl and gcc via yum before running the pip requirements install. once i got all of that installed before building the first time everything seems to work.

The python environment seems to be pretty 'sticky'.... annoying.

All good now, thanks for your help!

austinbyers commented 7 years ago

Happy to help - I'm glad you got it working! I'll leave the issue open until we update BinaryAlert's documentation accordingly.

What do you mean by the environment is 'sticky'?

crobo1337 commented 7 years ago

By sticky, I mean that even after nuking the virtual environment, and installing the correct packages on my build agent, any new virtual environments built after that are still building as if the openssl module isn't installed.

This behavior is sort of detailed here: https://github.com/VirusTotal/yara-python/issues/28

austinbyers commented 7 years ago

Oh weird. Again, thanks for letting us know!