[Security Solution] Exceptions don't work on keyword values containing a new line symbol

[maximpn]

Summary

It turns out exceptions don't work on a keyword field if it contains a new line symbol.

Steps to reproduce

• Index an event with a keyword field containing a new line symbol

POST /logs-test-1/_doc
{
  "@timestamp": <numeric timestamp>,
  "ecs": { "version": "8.0.0" },
  "host": {
    "name": "some\nvalue"
  },
  "event": {
    "kind": "event",
    "category": "iam",
    "type": ["group", "info"],
  }
}

• Create a custom query rule with *:* query
• Add an exception to just created rule

host.name IS "some\nvalue"

OR

host.name IS "some\\nvalue"

• Check rule preview (make sure selected time range covers event's timestamp)

ER: An indexed event doesn't lead to alert generation since added exception skips it. AR: There is an alert generated.

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-detection-engine (Team:Detection Engine)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[marshallmain]

It appears that this is a problem with the exceptions UI automatically escaping special characters, so \n in the exceptions UI is sent to the API as \\n. This makes it impossible to create an exception with only \n in the exceptions UI. If you try to create an exception through the UI with \n and retrieve it through dev tools to see what was actually created, you'll see something like

{
  "exception-list": {
    "comments": [],
    "created_at": "2024-04-15T17:34:43.769Z",
    "created_by": "marshall.main@elastic.co",
    "description": "Exception list item",
    "entries": [
      {
        "field": "host.name",
        "operator": "included",
        "type": "match",
        "value": """test\ntest"""
      }
    ],
    "item_id": "3467bc7e-90d8-4595-baf1-8690d2b35713",
    "list_id": "728ecca6-625c-4367-b0b4-29dc1669a4da",
    "list_type": "item",
    "name": "test2",
    "os_types": [],
    "tags": [],
    "tie_breaker_id": "642311d5-ce8e-47ef-9093-9dd68e366073",
    "type": "simple",
    "updated_by": "marshall.main@elastic.co",
    "version": null
  },
  "type": "exception-list",
  "references": [],
  "managed": false,
  "namespaces": [
    "default"
  ],
  "coreMigrationVersion": "8.8.0",
  "typeMigrationVersion": "8.0.0",
  "updated_at": "2024-04-15T18:35:10.614Z",
  "created_at": "2024-04-15T18:35:10.614Z"
}

Note the triple quotes around entries.value - per https://github.com/elastic/kibana/pull/9433, this means the value is being displayed without the extra escape characters.

You can use the exceptions API to create an exception without the extra escape characters. If you create an exception through the API with the value test\ntest, it shows up in dev tools instead as

"entries": [
  {
    "field": "host.name",
    "operator": "included",
    "type": "match",
    "value": """test
test"""
  }
],

and filters source documents that contain host.name: "test\ntest" as expected. The line break in the middle of entries.value with the triple quotes means entries.value is equivalent to test\ntest with single quotes.

I tried copy-pasting a line break character into the exceptions UI as well and the validation prevented me from saving the exception.