Rework rules which aren't bound to a field

thomaspatzke commented 5 years ago

Rules which aren't bound to fields should be reworked to search on fields as much as possible. These rules are problematic in various ways:

There are SIEM systems that rely on matching on fields.
Searching on many fields usually causes a performance penalty in backends that support this

Rules identified so far:

rules/web/web_source_code_enumeration.yml
rules/web/web_webshell_keyword.yml
rules/web/web_apache_threading_error.yml
rules/web/web_apache_segfault.yml
rules/windows/sysmon/sysmon_logon_scripts_userinitmprlogonscript.yml
rules/windows/other/win_wmi_persistence.yml
rules/windows/builtin/win_susp_msmpeng_crash.yml
rules/windows/builtin/win_alert_active_directory_user_control.yml
rules/windows/builtin/win_av_relevant_match.yml
rules/windows/builtin/win_mal_creddumper.yml
rules/windows/builtin/win_susp_sam_dump.yml
rules/windows/builtin/win_alert_mimikatz_keywords.yml
rules/windows/builtin/win_alert_enable_weak_encryption.yml
rules/windows/powershell/powershell_malicious_keywords.yml
rules/windows/powershell/powershell_suspicious_invocation_specific.yml
rules/windows/powershell/powershell_suspicious_keywords.yml
rules/windows/powershell/powershell_suspicious_download.yml
rules/windows/powershell/powershell_malicious_commandlets.yml
rules/application/app_sqlinjection_errors.yml
rules/application/appframework_django_exceptions.yml
rules/application/appframework_spring_exceptions.yml
rules/application/appframework_ruby_on_rails_exceptions.yml
rules/apt/apt_equationgroup_lnx.yml
rules/linux/lnx_shell_clear_cmd_history.yml
rules/linux/lnx_susp_ssh.yml
rules/linux/lnx_susp_named.yml
rules/linux/lnx_shell_priv_esc_prep.yml
rules/linux/lnx_shell_susp_rev_shells.yml
rules/linux/lnx_buffer_overflows.yml
rules/linux/lnx_sudo_cve_2019_14287.yml
rules/linux/lnx_ssh_cve_2018_15473.yml
rules/linux/lnx_susp_vsftp.yml
rules/linux/lnx_shell_susp_commands.yml
rules/linux/lnx_clamav.yml
rules/linux/lnx_shell_susp_log_entries.yml

Karneades commented 5 years ago

Would you like to change the title to "Rework rules which aren't bound to a field" so it matches with your description and not having that scary "remove" in it ;)

I'll try to work through some of them and will create PRs and link them so you're able to strike them through in your list above.

Karneades commented 5 years ago

Fixed rules

rules/windows/other/win_wmi_persistence.yml
rules/windows/powershell/powershell_malicious_commandlets.yml
rules/windows/powershell/powershell_malicious_keywords.yml
rules/windows/powershell/powershell_suspicious_download.yml
rules/windows/powershell/powershell_suspicious_invocation_specific.yml
rules/windows/powershell/powershell_suspicious_keywords.yml
rules/windows/powershell/powershell_prompt_credentials.yml
rules/windows/builtin/win_susp_msmpeng_crash.yml
rules/windows/builtin/win_alert_active_directory_user_control.yml
rules/windows/builtin/win_av_relevant_match.yml
rules/windows/builtin/win_mal_creddumper.yml
rules/windows/builtin/win_susp_sam_dump.yml
rules/windows/builtin/win_alert_mimikatz_keywords.yml
rules/windows/builtin/win_alert_enable_weak_encryption.yml
rules/windows/sysmon/sysmon_logon_scripts_userinitmprlogonscript.yml

New rules identified

rules/windows/powershell/powershell_prompt_credentials.yml
rules/application/app_python_sql_exceptions.yml

Issues

For some rules (like rules/linux/lnx_shell_susp_commands.yml) we first have to clarify which service or log source is used, sometimes only the product is given.
And for others like rules/linux/lnx_clamav.yml where the raw log file is analyzed for keywords and where no field extraction is used, I'm unsure how to fix it.

thomaspatzke commented 5 years ago

Would you like to change the title to "Rework rules which aren't bound to a field" so it matches with your description and not having that scary "remove" in it ;)

Oh! :grin: Just changed it!

I'll try to work through some of them and will create PRs and link them so you're able to strike them through in your list above.

Thanks a lot!

Karneades commented 5 years ago

@thomaspatzke I changed all the windows rules where we mostly know from existing rules which fields exist or which field is available by default - but for all the linux, application and web rules I don't know the used fields and if field extraction is used at all. Mostly the raw events are just indexed into log systems.

I unassign the issue and hope someone can jump in here to fix some of the remaining rules.

See https://github.com/Neo23x0/sigma/issues/501#issuecomment-547570317 for the progress of fixed rules and output of the work I've done so far.

frack113 commented 2 years ago

Sorry this post is closed automatically because it is not more active

SigmaHQ / sigma

Rework rules which aren't bound to a field #501