ESQL / Correlations: AttributeError when using siem_rule/kibana_ndjson formats

[13621]

Hi, I love this project and especially like the recent introduction of correlations. I wanted to try out different formats for the ESQL backend, and I think I found a bug related to the siem_rule and kibana_ndjson formats.

When converting a correlation rule to these formats in the ESQL backend, I get AttributeError: 'dict' object has no attribute 'split'. I used this code to test it out:

from sigma.collection import SigmaCollection
from sigma.backends.elasticsearch import ESQLBackend
from sigma.pipelines.elasticsearch import ecs_windows

# rule from https://sigmahq.io/docs/meta/correlations.html#event-count
rule = """
title: Windows Failed Logon Event
name: failed_logon
status: test
description: Detects failed logon events on Windows systems.
logsource:
  product: windows
  service: security
detection:
  selection:
    EventID: 4625
  filter:
    SubjectUserName|endswith: $
  condition: selection and not filter
---
title: Multiple failed logons for a single user (possible brute force attack)
status: test
correlation:
  type: event_count
  rules:
    - failed_logon
  group-by:
    - TargetUserName
    - TargetDomainName
  timespan: 5m
  condition:
    gte: 10
"""

coll = SigmaCollection.from_yaml(rule)
back = ESQLBackend(ecs_windows())
res = back.convert(coll, "siem_rule")  # AttributeError: 'dict' object has no attribute 'split'
res = back.convert(coll, "kibana_ndjson")  # AttributeError: 'dict' object has no attribute 'split'
print(res)

I found that I don't get this error when converting a normal rule to siem_rule or kibana_ndjson format, or converting a correlation rule to the default format (just the query) with the ESQL backend. PySigma and the elasticsearch backend are both on the most recent version (0.11.12 and 1.1.2 respectively).

[m4dh4t]

While working on a fix for this I stumbled upon another issue on which it is dependent, tracked here: https://github.com/SigmaHQ/pySigma/issues/277

A PR to fix it is awaiting feedback, once this is solved I will start working on this one.

[m4dh4t]

The PR has now been merged and I have a fix ready here. It causes a breaking change due to the way this backend handles correlations, so I'll have to wait for the next pySigma version to be released before submitting the PR.

[13621]

The PR has now been merged and I have a fix ready here. It causes a breaking change due to the way this backend handles correlations, so I'll have to wait for the next pySigma version to be released before submitting the PR.

Thank you for your fix 👍 Just tested this with the code I gave above. The errors seem to be resolved now with both formats working fine. You're right though, I had to build pySigma from the main branch so that the tests would pass. Would love a new release of pySigma so that this fix can be merged.

[m4dh4t]

Thank you for your feedback, nice to know it works as expected on your side too. I've opened the PR and hopefully the fix will be released soon https://github.com/SigmaHQ/pySigma-backend-elasticsearch/pull/82.