SigmaHQ / pySigma-backend-elasticsearch

pySigma Elasticsearch backend
GNU Lesser General Public License v3.0
42 stars 26 forks source link

[Correlations] ES|QL should perform a unique reduction on indexes #65

Closed sinnwise closed 2 months ago

sinnwise commented 4 months ago

On the following Correlation rule, I noticed that the ES|QL output has the index name twice, such that Elastic complains about running out of memory, probably because it's loading the index into memory twice over.

The main problem here is from okta-logs,okta-logs needs to end up being from okta-logs otherwise:

Screenshot 2024-06-24 at 15 16 17

Query:

from okta-logs,okta-logs
| where (okta.event_type=="user.session.start" and okta.outcome.result=="FAILURE") or (okta.event_type=="user.session.start" and okta.outcome.result=="SUCCESS")
| eval event_type=case(okta.event_type=="user.session.start" and okta.outcome.result=="FAILURE", "okta_failed_login", okta.event_type=="user.session.start" and okta.outcome.result=="SUCCESS", "okta_successful_login")
| eval timebucket=date_trunc(1hours, @timestamp) | stats event_type_count=count_distinct(event_type) by timebucket, okta.actor.alternate_id
| where event_type_count >= 1

Pipeline:

name: Elastic 8.x
transformations:
 # ... other transformations ...
  # Okta
  - id: okta_set_index
    type: set_state
    key: "index"
    val: "okta-logs"
    rule_conditions:
      - type: logsource
        service: okta
        product: okta

Rule:

title: Okta Failed Login
id: 91b76b84-8589-47aa-9605-c837583b82a9
name: okta_failed_login
status: experimental
logsource:
    product: okta
    service: okta
detection:
    selection:
        event_type: user.session.start
        outcome.result: FAILURE
    condition: selection
---
title: Okta Successful Login
id: 91b76b84-8589-47aa-9605-c837583b82a3
name: okta_successful_login
status: experimental
logsource:
    product: okta
    service: okta
detection:
    selection:
        event_type: user.session.start
        outcome.result: SUCCESS
    condition: selection
---
title: Okta Failed Login Followed by Successful Login
id: dc8a8b2a-f97a-4000-a530-ff5de0c23643
status: experimental
correlation:
    type: temporal
    rules:
        - okta_failed_login
        - okta_successful_login
    group-by:
        - okta.actor.alternate_id
    timespan: 1h
    condition:
        gte: 1
level: medium
falsepositives:
    - Users forgetting their password and then successfully logging in
    - Password resets followed by successful logins