search

Yamato-Security / sigma-to-hayabusa-converter

Documentation and tools to curate Sigma rules for Windows event logs into easier to parse rules.
GNU General Public License v3.0
2 stars 0 forks source link

feat: add support multiple yaml conversion #9

Closed fukusuket closed 1 month ago

fukusuket commented 2 months ago

What Changed

I would appreciate it if you could check it out when you have time🙏

fukusuket commented 2 months ago

Test

There is no diff before/after this PR.

% poetry run python sigma-to-hayabusa-converter.py -r ../sigma -o ./converted_sigma_rules_org  # main
...
[INFO:sigma-to-hayabusa-converter.py:624] [4254] files created successfully.[13] files failed to convert. Created files were saved under [./converted_sigma_rules].
[INFO:sigma-to-hayabusa-converter.py:626] Script took [44.14] seconds.
...
% poetry run python sigma-to-hayabusa-converter.py -r ../sigma -o ./converted_sigma_rules_new # This PR
...
[INFO:sigma-to-hayabusa-converter.py:624] [4254] files created successfully.[13] files failed to convert. Created files were saved under [./converted_sigma_rules].
[INFO:sigma-to-hayabusa-converter.py:626] Script took [44.14] seconds.
...
% diff -qr converted_sigma_rules_new converted_sigma_rules_org 
%
fukusuket commented 2 months ago

Test(service:security)

original rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: value_count
  rules:
    - rule-a
    - rule-b
  group-by:
    - Computer
  timespan: 5m
  condition:
    gte: 2
    field: SubStatus
---
title: Logon Failure (User Does Not Exist)
name: rule-a

logsource:
  product: windows
  service: security
detection:
  selection_basic:
    EventID: 4625
  selection_user_not_exist:
    SubStatus: "0xc0000064" #Username does not exist error
  condition: selection_basic and selection_user_not_exist
---
title: Logon Failure (Wrong Password)
name: rule-b

logsource:
  product: windows
  service: security
detection:
  selection_basic:
    EventID: 4625
  selection_wrong_password:
    SubStatus: "0xc000006a" #Wrong password
  condition: selection_basic and selection_wrong_password

converted rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: Logon Failure (User Does Not Exist)
name: rule-a
logsource:
    product: windows
    service: security
detection:
    security:
        Channel: Security
    selection_basic:
        EventID: 4625
    selection_user_not_exist:
        SubStatus: '0xc0000064' #Username does not exist error
    condition: security and (selection_basic and selection_user_not_exist)
ruletype: Sigma
---
title: Logon Failure (Wrong Password)
name: rule-b
logsource:
    product: windows
    service: security
detection:
    security:
        Channel: Security
    selection_basic:
        EventID: 4625
    selection_wrong_password:
        SubStatus: '0xc000006a' #Wrong password
    condition: security and (selection_basic and selection_wrong_password)
ruletype: Sigma
fukusuket commented 2 months ago

Test(service:process_creation)

original rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: value_count
  rules:
    - rule-a
    - rule-b
  group-by:
    - Computer
  timespan: 5m
  condition:
    gte: 2
    field: SubStatus
---
title: TEST1
name: rule-a

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\example.exe'
  condition: selection
---
title: TEST2
name: rule-b

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\test.exe'
  condition: selection

converted rule(Sysmon:1)

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \example.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \test.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma

converted rule(Security:4688)

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \example.exe
    condition: process_creation and selection
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \test.exe
    condition: process_creation and selection
ruletype: Sigma
YamatoSecurity commented 2 months ago

@fukusuket Thanks so much! I forgot about that with process_creation rules, etc... we need to create 2 new rules with different IDs. It looks like the ID is not changing after conversion. I guess we need to update the IDs of the referenced rules as well for Security 4688 events. What do you think?

fukusuket commented 2 months ago

@YamatoSecurity Thank you so much for checking :) It's true that the ID hasn't been updated... so I'll fix it!

fukusuket commented 2 months ago

original rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: value_count
  rules:
    - rule-a
    - rule-b
  group-by:
    - Computer
  timespan: 5m
  condition:
    gte: 2
    field: SubStatus
---
title: TEST1
name: rule-a

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\example.exe'
  condition: selection
---
title: TEST2
name: rule-b

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\test.exe'
  condition: selection

converted rule(Sysmon:1)

title: Value Count TEST
id: 19bca9b3-c751-3caa-f177-fa971a7a7a38
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \example.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \test.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma

converted rule(Security:4688)

title: Value Count TEST
id: e5c0e1b2-b7ce-8870-2db4-fe3ef3a00bdb
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \example.exe
    condition: process_creation and selection
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \test.exe
    condition: process_creation and selection
ruletype: Sigma

@YamatoSecurity I fixed to set a new UUID! Could you confirm it?🙏

YamatoSecurity commented 1 month ago

@fukusuket Thanks so much! In these examples, we are using name: rule-a and name: rule-b to reference the other rules but it is also possible to use IDs instead. What will happen to the IDs of the converted rules in that case? Are we calculating new rule IDs for the Security 4688 referenced rules?

fukusuket commented 1 month ago

@YamatoSecurity Thank you so much for checking :) I had forgotten about the above case ...😇 I will implement this additionally!

fukusuket commented 1 month ago

UUID rule

original rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: value_count
  rules:
    - e87bd730-df45-4ae9-85de-6c75369c5d29
    - 8afa97ce-a217-4f7c-aced-3e320a57756d
  group-by:
    - Computer
  timespan: 5m
  condition:
    gte: 2
    field: SubStatus
---
title: TEST1
name: rule-a
id: 8afa97ce-a217-4f7c-aced-3e320a57756d

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\example.exe'
  condition: selection
---
title: TEST2
name: rule-b
id: e87bd730-df45-4ae9-85de-6c75369c5d29

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\test.exe'
  condition: selection

converted rule(sysmon:1)

title: Value Count TEST
id: 19bca9b3-c751-3caa-f177-fa971a7a7a38
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - cf3887ec-333c-8596-6ba3-900ac974ec13
        - 868de26e-04e9-af4d-d254-c807d97611e9
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
id: cf3887ec-333c-8596-6ba3-900ac974ec13
name: rule-a
related:
    - id: 8afa97ce-a217-4f7c-aced-3e320a57756d
      type: derived
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \example.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma
---
title: TEST2
id: 868de26e-04e9-af4d-d254-c807d97611e9
name: rule-b
related:
    - id: e87bd730-df45-4ae9-85de-6c75369c5d29
      type: derived
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \test.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma

converted rule(security:4688)

title: Value Count TEST
id: e5c0e1b2-b7ce-8870-2db4-fe3ef3a00bdb
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - 409b7755-bb8b-4e7a-2bcf-1974ec769fca
        - 488bd8bc-37a5-8355-0a27-f6c6882d15d0
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
id: 409b7755-bb8b-4e7a-2bcf-1974ec769fca
name: rule-a
related:
    - id: 8afa97ce-a217-4f7c-aced-3e320a57756d
      type: derived
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \example.exe
    condition: process_creation and selection
ruletype: Sigma
---
title: TEST2
id: 488bd8bc-37a5-8355-0a27-f6c6882d15d0
name: rule-b
related:
    - id: e87bd730-df45-4ae9-85de-6c75369c5d29
      type: derived
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \test.exe
    condition: process_creation and selection
ruletype: Sigma
fukusuket commented 1 month ago

Name rule

original rule

title: Value Count TEST
id: 0e95725d-7320-415d-80f7-004da920fc11
level: informational
correlation:
  type: value_count
  rules:
    - rule-a
    - rule-b
  group-by:
    - Computer
  timespan: 5m
  condition:
    gte: 2
    field: SubStatus
---
title: TEST1
name: rule-a

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\example.exe'
  condition: selection
---
title: TEST2
name: rule-b

logsource:
    category: process_creation
    product: windows
detection:
  selection:
    Image|endswith: '\test.exe'
  condition: selection

converted rule(sysmon:1)

title: Value Count TEST
id: 19bca9b3-c751-3caa-f177-fa971a7a7a38
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \example.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 1
        Channel: Microsoft-Windows-Sysmon/Operational
    selection:
        Image|endswith: \test.exe
    condition: process_creation and selection
tags:
    - sysmon
ruletype: Sigma

converted rule(security:4688)

title: Value Count TEST
id: e5c0e1b2-b7ce-8870-2db4-fe3ef3a00bdb
related:
    - id: 0e95725d-7320-415d-80f7-004da920fc11
      type: derived
level: informational
correlation:
    type: value_count
    rules:
        - rule-a
        - rule-b
    group-by:
        - Computer
    timespan: 5m
    condition:
        gte: 2
        field: SubStatus
ruleType: Sigma
---
title: TEST1
name: rule-a
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \example.exe
    condition: process_creation and selection
ruletype: Sigma
---
title: TEST2
name: rule-b
logsource:
    category: process_creation
    product: windows
detection:
    process_creation:
        EventID: 4688
        Channel: Security
    selection:
        NewProcessName|endswith: \test.exe
    condition: process_creation and selection
ruletype: Sigma
fukusuket commented 1 month ago

@YamatoSecurity I fixed https://github.com/Yamato-Security/sigma-to-hayabusa-converter/pull/9#issuecomment-2186010010 case! Could you confirm it?🙏