Extended transformation condition logic options

[joshnck]

I have a current issue where I am splitting the index from my EDR to pull out process creation and process end events out of the primary index. Previously I could just say "if product = windows, apply edr index" but now I need to say "if product = windows and category = process_creation, apply edr_process_start index, else apply edr index"

Unfortunately, there is not a clear way to do this without explicitly calling out every category and assigning the edr index. My first thought was to just use the processing_item_applied and if the process creation item is not applied, then apply the else. Logically this made sense but then I came into another problem - this would change the condition addition state for EVERY rule converted.

My proposed solution is to support complex rule condition logic. Currently we support rule_cond_not which will negate the entire set of rule conditions, but we do not have the ability to negate specific rule condition items nor do we have the ability to perform complex logic. If we implemented a rule_cond_logic object that performed in the same way that Sigma rules hand condition logic, we could solve this problem and allow ourselves some flexibility for significantly more complex transformations. (Someone may want a transformation that says "(cond1 or cond2) and (cond3 or cond4)" for example.

To implement this we will need two functions:

1. Condition objects need an ID field. This can be an optional field that is only used when rule_cond_logic is used
2. Implementation of rule_cond_logic which replaces rule_cond_op and has the default state of and but has the ability to be overwritten such that specific condition object IDs can be called with logic applied.

The final transformation yaml object could look like this:

name: edr
priority: 20
transformations:

#### INDEX AND OS ASSIGNMENTS
  - id: edr_process_creation
    type: add_condition
    conditions:
      index: "edr_process_start"
    rule_conditions:
      - type: logsource
        product: windows
        category: process_creation

  - id: edr_index
    type: add_condition
    conditions:
      index: 
        - "edr"
    rule_conditions:
      - type: logsource
        id: logsource_cond
        product: windows
      - type: processing_item_applied
        id: proc_creation_cond
        processing_item_id: edr_process_creation
    rule_cond_logic: logsource_cond and not proc_creation_cond

[thomaspatzke]

Good idea! I think this could even be realized in both ways and keep existing pipelines downward-compatible by allowing both mutually exclusive: the simplified rule_cond_op/rule_cond_negation and a rule_cond_logic with a full-blown condition.

[thomaspatzke]

Just merged the branch implementing processing condition expressions. It can be used by adding a *_cond_expr item in YAML piplelines, e.g. rule_cond_expr. The condition items in the *_cond items must then be specified named as map and can then referred by this name from the expression. Here are the docs.

Your example then ends up as follows:

    rule_conditions:
      logsource_cond:
        type: logsource
        product: windows
      logsource_cond:
        type: processing_item_applied
        processing_item_id: edr_process_creation
    rule_cond_expr: logsource_cond and not proc_creation_cond

It is fully downwards compatible, existing pipelines are not affected.

I will release it soon along with some other contributions.