Some `Sigma.Windows.Hayabusa.Rule` rules does not detect logs

fukusuket commented 5 months ago

Hello :) I am trying Sigma.Windows.Hayabusa.Rules on 0.72 RC1 and some rules were not working, so I would like to report.(I noticed this when I was comparing the results with Exchange.Windows.EventLogs.Hayabusa)

Although I have only confirmed this on medium level or higher, the following rules did not detect logs in Sigma.Windows.Hayabusa.Rules.

Defender Alert (High)
A Rule Has Been Deleted From The Windows Firewall Exception List
~MSI Installation From Suspicious Locations~ ... Maybe this can be resolved by using the latest rule?
~Suspicious Non PowerShell WSMAN COM Provider~ ... Maybe this can be resolved by using the latest rule?
~Uncommon AppX Package Locations~... Maybe this can be resolved by using the latest rule?

Thank you for your time.

fukusuket commented 5 months ago

Defender Alert (High)

sigma_rules.yaml

title: Defender Alert (High)
logsource:
  product: windows
  service: windefend
detection:
  condition: []
  selection:
    Channel: Microsoft-Windows-Windows Defender/Operational
    EventID: 1116
    SeverityID: 4
status: test
author: Zach Mathis, Fukusuke Takahashi
level: high
references:
  - https://github.com/Yamato-Security/hayabusa-rules/tree/main/hayabusa/hayabusa/builtin/WindowsDefender/Defender_1116_High_Alert.yml

Hayabusa original

author: Zach Mathis, Fukusuke Takahashi
date: 2021/12/01
modified: 2023/6/17

title: 'Defender Alert (High)'
description: Windows defender malware detection

id: 1e11c0f0-aecd-45d8-9229-da679c0265ea
level: high
status: test
logsource:
    product: windows
    service: windefend
detection:
    selection:
        Channel: Microsoft-Windows-Windows Defender/Operational
        EventID: 1116
        SeverityID: 4 # High
falsepositives:
    - bad signature
tags:
    - malware
references:
    - https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/troubleshoot-microsoft-defender-antivirus?view=o365-worldwide
ruletype: Hayabusa

detected log sample

{
    "Timestamp": "2024-01-27T23:44:23.601016Z",
    "RuleTitle": "Defender Alert (High)",
    "Level": "high",
    "Computer": "mouse",
    "Channel": "Defender",
    "EventID": 1116,
    "RecordID": 2894,
    "Details": {
        "Threat": "HackTool:PowerShell/Mimikatz",
        "Severity": "高",
        "Type": "ツール",
        "User": "mouse\\fukus",
        "Path": "file:_C:\\Users\\fukus\\velociraptor-docs\\content\\knowledge_base\\tips\\decimaldecode.md",
        "Proc": "C:\\Users\\fukus\\AppData\\Local\\Programs\\Microsoft VS Code\\resources\\app\\node_modules.asar.unpacked\\@vscode\\ripgrep\\bin\\rg.exe"
    },
    "ExtraFieldInfo": {
        "Action ID": 9,
        "Action Name": "該当なし",
        "Additional Actions ID": 0,
        "Additional Actions String": "No additional actions required",
        "Category ID": 34,
        "Detection ID": "{7F1E0949-2ACC-4813-840F-B58BECE84913}",
        "Detection Time": "2024-01-27T23:44:23.284Z",
        "Engine Version": "AM: 1.1.23110.2, NIS: 1.1.23110.2",
        "Error Code": "0x00000000",
        "Error Description": "この操作を正しく終了しました。",
        "Execution ID": 1,
        "Execution Name": "中断",
        "FWLink": "https://go.microsoft.com/fwlink/?linkid=37020\u0026name=HackTool:PowerShell/Mimikatz\u0026threatid=2147725066\u0026enterprise=0",
        "Origin ID": 1,
        "Origin Name": "ローカル コンピューター",
        "Post Clean Status": 0,
        "Pre Execution Status": 0,
        "Product Name": "Microsoft Defender ウイルス対策",
        "Product Version": "4.18.23110.3",
        "Remediation User": "",
        "Security intelligence Version": "AV: 1.403.2791.0, AS: 1.403.2791.0, NIS: 1.403.2791.0",
        "Severity ID": 4,
        "Source ID": 3,
        "Source Name": "リアルタイム保護",
        "State": 1,
        "Status Code": 1,
        "Status Description": "",
        "Threat ID": 2147725066,
        "Type ID": 0,
        "Type Name": "コンクリート",
        "Unused2": "",
        "Unused3": "",
        "Unused4": "",
        "Unused5": "",
        "Unused6": "",
        "Unused": ""
    },
    "EventTime": "2024-01-27T23:44:23.601016Z"
}

fukusuket commented 5 months ago

A Rule Has Been Deleted From The Windows Firewall Exception List

sigma_rules.yaml

title: A Rule Has Been Deleted From The Windows Firewall Exception List
logsource:
  product: windows
  service: firewall-as
detection:
  condition: (firewall_as and (selection and not 1 of filter_main_* and not 1 of filter_optional_*))
  filter_main_empty:
    ModifyingApplication: ""
  filter_main_generic:
    ModifyingApplication|startswith:
      - C:\Program Files\
      - C:\Program Files (x86)\
  filter_main_null:
    ModifyingApplication: null
  filter_main_svchost:
    ModifyingApplication: C:\Windows\System32\svchost.exe
  filter_optional_msmpeng:
    ModifyingApplication|startswith: C:\ProgramData\Microsoft\Windows Defender\Platform\
    ModifyingApplication|endswith: \MsMpEng.exe
  firewall_as:
    Channel: Microsoft-Windows-Windows Firewall With Advanced Security/Firewall
  selection:
    EventID:
      - 2006
      - 2052
status: experimental
author: frack113
level: medium
references:
  - https://github.com/Yamato-Security/hayabusa-rules/tree/main/hayabusa/sigma/builtin/firewall_as/win_firewall_as_delete_rule.yml

Hayabusa original

title: A Rule Has Been Deleted From The Windows Firewall Exception List
id: c187c075-bb3e-4c62-b4fa-beae0ffc211f
status: experimental
description: Detects when a single rules or all of the rules have been deleted from the Windows Defender Firewall
references:
    - https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-r2-and-2008/dd364427(v=ws.10)
author: frack113
date: 2022/02/19
modified: 2023/06/12
tags:
    - attack.defense_evasion
    - attack.t1562.004
logsource:
    product: windows
    service: firewall-as
detection:
    firewall_as:
        Channel: Microsoft-Windows-Windows Firewall With Advanced Security/Firewall
    selection:
        EventID:
            - 2006 # A rule has been deleted in the Windows Defender Firewall exception list
            - 2052 # A rule has been deleted in the Windows Defender Firewall exception list. (Windows 11)
    filter_main_generic:
        ModifyingApplication|startswith:
            - C:\Program Files\
            - C:\Program Files (x86)\
    filter_main_svchost:
        ModifyingApplication: C:\Windows\System32\svchost.exe
    filter_optional_msmpeng:
        ModifyingApplication|startswith: C:\ProgramData\Microsoft\Windows Defender\Platform\
        ModifyingApplication|endswith: \MsMpEng.exe
    filter_main_null:
        ModifyingApplication:
    filter_main_empty:
        ModifyingApplication: ''
    condition: firewall_as and (selection and not 1 of filter_main_* and not 1 of filter_optional_*)
level: medium
ruletype: Sigma

detected log sample

{
    "Timestamp": "2024-03-13T22:05:25.126032Z",
    "RuleTitle": "A Rule Has Been Deleted From The Windows Firewall Exception List",
    "Level": "med",
    "Computer": "mouse",
    "Channel": "Firewall",
    "EventID": 2052,
    "RecordID": 12812,
    "Details": {},
    "ExtraFieldInfo": {
        "ErrorCode": 0,
        "ModifyingApplication": "C:\\Windows\\SystemTemp\\chrome_Unpacker_BeginUnzipping4360_1830454617\\CR_8BAC2.tmp\\setup.exe",
        "ModifyingUser": "S-1-5-18",
        "RuleId": "{D7B81251-9069-467C-A54A-3AD41CE559FC}",
        "RuleName": "b380f8ff-020d-464a-ad92-63d548cfc877"
    },
    "EventTime": "2024-03-13T22:05:25.126032Z"
}

scudette commented 5 months ago

Thank you so much for testing and reporting these issues!

I looked at the windows defender rule and the reason it is not triggering is because it uses the field SeverityID - in the config file we map SeverityID to EventData.Severity ID here

https://github.com/Velocidex/velociraptor-sigma-rules/blob/7271bb519c4ae0eaf7b6340e2f3651074f7804a6/config/windows_hayabusa_rules.yaml#L238

Based on the Hayabusa aliases file: https://github.com/Yamato-Security/hayabusa-rules/blob/09dba13950cb849d320d76ae63707496c5947f14/config/eventkey_alias.txt#L159

But looking at the actual docs from Microsoft there is no such field: https://github.com/Yamato-Security/hayabusa-rules/blob/09dba13950cb849d320d76ae63707496c5947f14/config/eventkey_alias.txt#L159

The field is actually called Severity (without the ID).

This may just be me misunderstand how Hayabusa maps the fields but this seems to be a bug in the Hayabusa event map?

scudette commented 5 months ago

I also noticed another bug which might also be my misunderstanding of the Sigma format - the generated rule has no conditions (conditions: []) I was unaware that it is possible to have a rule without a condition clause - we are not checking for this so we end up generating an empty condition which will not match anything

fukusuket commented 5 months ago

@scudette Thank you for checking! The following is the XML obtained from the actual evtx, but since the Severity ID exists(FYI: https://github.com/Yamato-Security/hayabusa-rules/issues/349), I think the information in the Microsoft document is probably outdated ...🤔

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
  <Provider Name="Microsoft-Windows-Windows Defender" Guid="{11cd958a-c507-4ef3-b3f2-5fd9dfbd2c78}" /> 
  <EventID>1116</EventID> 
  <Version>0</Version> 
  <Level>3</Level> 
  <Task>0</Task> 
  <Opcode>0</Opcode> 
  <Keywords>0x8000000000000000</Keywords> 
  <TimeCreated SystemTime="2024-03-20T00:20:22.7423487Z" /> 
  <EventRecordID>4811</EventRecordID> 
  <Correlation /> 
  <Execution ProcessID="5284" ThreadID="11304" /> 
  <Channel>Microsoft-Windows-Windows Defender/Operational</Channel> 
  <Computer>mouse</Computer> 
  <Security UserID="S-1-5-18" /> 
  </System>
<EventData>
  <Data Name="Product Name">Microsoft Defender ウイルス対策</Data> 
  <Data Name="Product Version">4.18.24020.7</Data> 
  <Data Name="Detection ID">{03A695D0-DCC9-4237-942D-B3B1FE296A77}</Data> 
  <Data Name="Detection Time">2024-03-20T00:20:22.660Z</Data> 
  <Data Name="Unused" /> 
  <Data Name="Unused2" /> 
  <Data Name="Threat ID">2147720558</Data> 
  <Data Name="Threat Name">TrojanDownloader:PowerShell/Plasti.A</Data> 
  <Data Name="Severity ID">5</Data> 
  <Data Name="Severity Name">重大</Data> 
  <Data Name="Category ID">4</Data> 
  <Data Name="Category Name">ダウンローダー型のトロイの木馬</Data> 
  <Data Name="FWLink">https://go.microsoft.com/fwlink/?linkid=37020&name=TrojanDownloader:PowerShell/Plasti.A&threatid=2147720558&enterprise=0</Data> 
  <Data Name="Status Code">1</Data> 
  <Data Name="Status Description" /> 
  <Data Name="State">1</Data> 
  <Data Name="Source ID">3</Data> 
  <Data Name="Source Name">リアルタイム保護</Data> 
  <Data Name="Process Name">C:\tmp\takajo-2.4.0-win\takajo.exe</Data> 
  <Data Name="Detection User">mouse\fukus</Data> 
  <Data Name="Unused3" /> 
  <Data Name="Path">file:_C:\tmp\takajo-2.4.0-win\case-1\StackServices.csv</Data> 
  <Data Name="Origin ID">1</Data> 
  <Data Name="Origin Name">ローカル コンピューター</Data> 
  <Data Name="Execution ID">1</Data> 
  <Data Name="Execution Name">中断</Data> 
  <Data Name="Type ID">0</Data> 
  <Data Name="Type Name">コンクリート</Data> 
  <Data Name="Pre Execution Status">0</Data> 
  <Data Name="Action ID">9</Data> 
  <Data Name="Action Name">該当なし</Data> 
  <Data Name="Unused4" /> 
  <Data Name="Error Code">0x00000000</Data> 
  <Data Name="Error Description">この操作を正しく終了しました。</Data> 
  <Data Name="Unused5" /> 
  <Data Name="Post Clean Status">0</Data> 
  <Data Name="Additional Actions ID">0</Data> 
  <Data Name="Additional Actions String">No additional actions required</Data> 
  <Data Name="Remediation User" /> 
  <Data Name="Unused6" /> 
  <Data Name="Security intelligence Version">AV: 1.407.561.0, AS: 1.407.561.0, NIS: 1.407.561.0</Data> 
  <Data Name="Engine Version">AM: 1.1.24020.9, NIS: 1.1.24020.9</Data> 
  </EventData>
  </Event>

scudette commented 5 months ago

Yes you are correct! I just generated a similar event on a live system. This looks to be an issue of us not handling a missing condition field properly - I could not determine from https://sigmahq.io/docs/basics/rules.html#detection if it is event allowed to omit the condition clause.

I suppose we can just add one in case.

fukusuket commented 5 months ago

Yes, it is not clear from the specifications whether the condition clause is required... I will check whether it is better to add the condition clause on the Hayabusa rule side!

fukusuket commented 5 months ago

It seems that the condition section was probably omitted (because other rules do not omit the condition clause ), so I fixed it with the PR below. Sorry for our mistake!

https://github.com/Yamato-Security/hayabusa-rules/pull/619

fukusuket commented 5 months ago

The following detection may also be a problem on Hayabusa's(or Sigma) side, so I will check it.... https://github.com/Velocidex/velociraptor-sigma-rules/issues/24#issuecomment-2008471759

fukusuket commented 5 months ago

The following detection may also be a problem on Hayabusa's(or Sigma) side, so I will check it.... https://github.com/Velocidex/velociraptor-sigma-rules/issues/24#issuecomment-2008471759

In the above case, Hayabusa was able to detect the logs as expected :)

scudette commented 5 months ago

Looking closer at the firewall rule above I get the following error from the engine:

[INFO] 2020-05-31T15:28:05Z Velociraptor: DEFAULT:While evaluating rule A Rule Has Been Deleted From The Windows Firewall Exception List: error evaluating search filter_main_null: expected scalar field matching value got: <nil> (<nil>)

That search is

    filter_main_null:
        ModifyingApplication:

Im not really sure what its supposed to say here? Is it meant to match the empty string?

scudette commented 5 months ago

Actually in the version of the rule we use it actually says null

https://github.com/Yamato-Security/hayabusa-rules/blob/09dba13950cb849d320d76ae63707496c5947f14/sigma/builtin/firewall_as/win_firewall_as_delete_rule.yml#L34

    filter_main_null:
        ModifyingApplication: null

Do you know what it's supposed to mean?

fukusuket commented 5 months ago

Yes, the null rule above is the correct rule. (Although it is not directly related to this issue, there was a problem where null was not output in the latest hayabusa rule... https://github.com/Yamato-Security/hayabusa-rules/issues/620)

The specifications of null in Sigma are as follows. https://github.com/SigmaHQ/sigma-specification/blob/main/Sigma_specification.md#special-field-values According to the specifications, empty strings and null seem to be distinguished. I think it indicates that the field exists but there is no value ... ?

fukusuket commented 5 months ago

(Although it is not directly related to this issue, there was a problem where null was not output in the latest hayabusa rule... https://github.com/Yamato-Security/hayabusa-rules/issues/620)

The bug which null was not output will be fixed in the following PR https://github.com/Yamato-Security/hayabusa-rules/pull/621.

fukusuket commented 4 months ago

It looks like the latest Hayabusa rules have been merged, so I'll close this issue and check the latest version! Thank you for your time :)

Velocidex / velociraptor-sigma-rules