Adding Yara Rules

[hackdefendr]

Problem: Using the YARA worker does not seem to work. Results are always blank.

In looking at the stoq.yar file it seems to be an index yara that is including the alienvault and fireeye yara files. So to add more yara rules, is it just as simple as editing stoq.yar and adding the new entries?

For example, if I clone the main yara-rules repo, can I just replace the index.yar from there and have stoq use them instead?

[mlaferrera]

Hi @hackdefendr -- You can certainly add more yara rules (or a new include statement with a path to your yara rules) to stoq.yar. You can also define rules by setting worker_rules in with stoq.cfg or yarascan.stoq, or at the command line by adding --plugin-opts yara:worker_rules=/path/to/rules.yar. You can find more information on plugin configuration options in the documention here.

[hackdefendr]

Follow up question(s)...

Seems that the yara-rules repo index files don't work. I changed my stoq.cfg to point to just the malware_index.yar and it errors stating one of the yara files could not be opened. I viewed the index, and the file it complains about, but don't see any issues.

[2020-01-28 13:15:17,876 ERROR] stoq: Worker plugin yara failed to load
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/stoq/core.py", line 812, in _resolve_plugin_dependencies
    plugin: WorkerPlugin = self.load_plugin(plugin_name)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/stoq/plugin_manager.py", line 177, in load_plugin
    plugin = plugin_class(plugin_config)
  File "/home/jsinglet/.stoq/plugins/yarascan/yarascan.py", line 59, in __init__
    self.worker_rules = self._compile_rules(worker_ruleset)
  File "/home/jsinglet/.stoq/plugins/yarascan/yarascan.py", line 93, in _compile_rules
    return yara.compile(filepath=filepath)
yara.SyntaxError: /home/jsinglet/.stoq/rules/./malware/MALW_AZORULT.yar(383): can't open include file: ./malware/TOOLKIT_exe2hex_payload.yar

Not really sure why it can't open the file.

"errors": [
        {
            "error": "Worker plugin yara failed to load: File \"/home/jsinglet/.stoq/plugins/yarascan/yarascan.py\", line 93, in _compile_rules ; yara.SyntaxError: /home/jsinglet/.stoq/rules/./malware/MALW_AZORULT.yar(383): can't open include file: ./malware/TOOLKIT_exe2hex_payload.yar",
            "plugin_name": "yara",
            "payload_id": "082873c2-1c4d-48db-95bf-fedbc61f6576"
        }
    ],

[mlaferrera]

It looks like your path to your rule files may be invalid. stoQ is looking for the yara file in /home/jsinglet/.stoq/rules/./malware/MALW_AZORULT.yar

[hackdefendr]

I cloned that yara-rules repo into /home/jsinglet/.stoq/rules so that path is correct.

My stoq.cfg:

[yara]
worker_rules = /home/jsinglet/.stoq/rules/malware_index.yar
dispatch_rules = rules/dispatcher.yar
strings_limit = 10
timeout = 60

That dot directory is being added during execution. Should I change the index entries to a relative path by removing the ./ ?

[mlaferrera]

ahh, it looks like that the path is being included (can't open include file: ./malware/TOOLKIT_exe2hex_payload.yar). You'll have to ensure the relative paths are accurate. Removing the ./ may fix the problem.

[hackdefendr]

Removing the ./ did not work. I tried copying just the malware yara rules into the plugin rules folder, and modified the stoq.yar file to include the malware_index.yar. Same error, different file:

[2020-01-28 13:38:50,519 ERROR] stoq: Worker plugin yara failed to load
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/stoq/core.py", line 812, in _resolve_plugin_dependencies
    plugin: WorkerPlugin = self.load_plugin(plugin_name)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/stoq/plugin_manager.py", line 177, in load_plugin
    plugin = plugin_class(plugin_config)
  File "/home/jsinglet/.stoq/plugins/yarascan/yarascan.py", line 59, in __init__
    self.worker_rules = self._compile_rules(worker_ruleset)
  File "/home/jsinglet/.stoq/plugins/yarascan/yarascan.py", line 93, in _compile_rules
    return yara.compile(filepath=filepath)
yara.SyntaxError: /home/jsinglet/.stoq/plugins/yarascan/rules/./malware/APT_APT1.yar(382): can't open include file: malware/TOOLKIT_Wineggdrop.yar

Both files exist and are readable. Something seems off when including multiple indexes.

When I add just a yara file to the stoq.yar it reads it in without issue.

Spoke to soon, when I add multiple yar files to the stoq.yar it bombs the same way.

[mlaferrera]

I'm not sure what the directory structure is so it is a bit hard to troubleshoot from my end. Yara is raising the exception because it cannot find the referenced rules. In the last error, it appears that the file malware/TOOLKIT_Wineggdrop.yar cannot be found relative to the file /home/jsinglet/.stoq/plugins/yarascan/rules/./malware/APT_APT1.yar.

[hackdefendr]

This is definitely weird. Its almost like it ignores the last line.

My directory structure is default for the most part, using ~/.stoq for home, and placing the additional yara rules inside the relative rules folder: /home/jsinglet/.stoq/plugins/yarascan/rules

All the additional rules are in: /home/jsinglet/.stoq/plugins/yarascan/rules/malware

[mlaferrera]

Closed due to inactivity. Please feel free to create a new issue if needed.