Yara analyzer: 'can't open include file'

[Tux-Panik]

Request Type

Bug / Support

Work Environment

(replace with N/A if not applicable)

Question	Answer
OS version (server)	Docker (on CentOS)
OS version (client)	7
Cortex Analyzer Name	Yara
Cortex Analyzer Version	Current (e3b618cbc15d196c2385e3e33a5f356b45bb0796)
Cortex Version	2.1.1-1
Browser type & version	N/A

Description

All jobs running Yara analyzer raised an error; while the mentioned file exists. Due to the error message, I guess it could be a location issue or something like this; but I still can't find the root cause.

Invalid output
Traceback (most recent call last):
  File "Yara/yara_analyzer.py", line 71, in <module>
    YaraAnalyzer().run()
  File "Yara/yara_analyzer.py", line 30, in __init__
    self.ruleset.append(yara.compile(rulepath + '/index.yar'))
yara.SyntaxError: /opt/Cortex-Analyzers/analyzers/Yara/rules/./malware/MALW_AZORULT.yar(431): can't open include file: ./malware/TOOLKIT_exe2hex_payload.yar

Steps to Reproduce

(keep this section only if the issue relates to a bug)

1. Adapt your 'docker-compose.yml' file to mount the 'Cortex-Analyzers' and the Yara rules folder as a volume
2. Enable the 'Yara' analyzer and adapt the configuration
3. Download the rules from https://github.com/Yara-Rules/rules on your host and in the folder that will fit the previous configuration
4. Run a Yara analyze job using a common file

Complementary information

From the Cortex's container, the first file exists. The second one is not found form the default path, but it is present if you are in the Yara rules folder (here: /opt/Cortex-Analyzers/analyzers/Yara/rules/). Perhaps this issue is due to the path from where the Yara analyzer is launch.

daemon@2e6b016f80d2:/opt/cortex$ ls -l /opt/Cortex-Analyzers/analyzers/Yara/rules/./malware/MALW_AZORULT.yar
-rwxrwxrwx 1 1000 1000 1427 Oct 15 15:28 /opt/Cortex-Analyzers/analyzers/Yara/rules/./malware/MALW_AZORULT.yar
daemon@2e6b016f80d2:/opt/cortex$
daemon@2e6b016f80d2:/opt/cortex$
daemon@2e6b016f80d2:/opt/cortex$ ls -l ./malware/TOOLKIT_exe2hex_payload.yar
ls: cannot access './malware/TOOLKIT_exe2hex_payload.yar': No such file or directory

Thank you for your help, Kind regards, Julien M.

[3c7]

Need to take a look on it. You can replace the relative paths in the MALW_AZORULT.yar with absolute paths, though, that should fix it.

[Tux-Panik]

Hi, thanks for your feedback. You are right, and I already did it in all '_index.yar' files, by replacing "./" by "/opt/Cortex-Analyzers/analyzers/rules/" but I missed the simply "index.yar" :-(

Now it works, and people can use the following command line: find /full/path/to/analyzers/Yara/rules/ -type f -iname "*index*.yar" -exec sed -i 's/\.\//\/opt\/Cortex-Analyzers\/analyzers\/Yara\/rules\//g' {} \;

However, the rule 'MALW_AZORULT.yar' raised an error on my side: SyntaxError: MALW_AZORULT.yar(23): invalid field name "sync" Removing this rule, and also all its inclusion, the analyzers works perfectly.

I used the following command, as I need to automate this action: for i in `grep -H -R "MALW_AZORULT\.yar" /full/path/to/analyzers/Yara/rules/ | grep -Eio "^.+\.yar:" | sed 's/:$//' | sort -u`; do sed -i '/MALW_AZORULT\.yar/d' $i; done

Probably dirty, but it works.

Last point: where do you grab your relevant (and compatible) Yara rules?

• https://github.com/Yara-Rules/rules
• ???

Kind regards, Julien

[3c7]

I found the yara-rules repository to be a mess, but maybe it's just me. Using the repository by florian roth often: https://github.com/Neo23x0/signature-base

[Tux-Panik]

I believe these repository has some fields/variables unsupported by someYara engines: https://github.com/viper-framework/viper/issues/545

[Tux-Panik]

I tested the one of @Neo23x0 and it fails:

Invalid output
Traceback (most recent call last):
  File "Yara/yara_analyzer.py", line 71, in <module>
    YaraAnalyzer().run()
  File "Yara/yara_analyzer.py", line 30, in __init__
    self.ruleset.append(yara.compile(rulepath + '/index.yar'))
yara.SyntaxError: /opt/Cortex-Analyzers/analyzers/Yara/signature-base/yara/yara_mixed_ext_vars.yar(308): undefined identifier "filename"

So, I renamed all "filename" to "_filename" and I reached the same error:

Invalid output
Traceback (most recent call last):
  File "Yara/yara_analyzer.py", line 71, in <module>
    YaraAnalyzer().run()
  File "Yara/yara_analyzer.py", line 30, in __init__
    self.ruleset.append(yara.compile(rulepath + '/index.yar'))
yara.SyntaxError: /opt/Cortex-Analyzers/analyzers/Yara/signature-base/yara/yara_mixed_ext_vars.yar(308): undefined identifier "file_name"

Regards,

[Neo23x0]

README says read me 😄

[Tux-Panik]

You are absolutely right. I guess this section is new, perhaps following the issue I mentioned above.

I also enjoy your reactivity :-) Feedback expected tomorrow

Regards, Julien

[Tux-Panik]

Confirmed. It works like a charm... As mentioned in the Cortex documentation, it is mandatory to create manually an "index.yar" file who contains all the rule files:

cd ./signature-base/yara/
rm -f ./generic_anomalies.yar ./general_cloaking.yar ./thor_inverse_matches.yar ./yara_mixed_ext_vars.yar
for i in `ls $(pwd)`; do echo "include \"/opt/Cortex-Analyzers/analyzers/Yara/signature-base/yara/$i\"" >> index.yar; done

Thanks guys, Regards, Julien