elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.16k forks source link

[Security Solution] `kibana.*` fields from the audit log are not added to detection alerts #184345

Open banderror opened 4 months ago

banderror commented 4 months ago

Summary

User is ingesting a Kibana Audit Log written to /usr/share/kibana/logs/kibana_audit.json back to Elasticsearch, and they want to set up some detection rules that would be generating alerts based on events from the audit log.

Some audit log events contain information about a rule being updated, including its saved object id:

    "message": "User is updating rule [id=1758f040-65cc-11ed-867e-318c4bd4f121]",
    "event": {
      "kind": "event",
      "module": "kibana",
      "action": "rule_update"
    },
    "kibana": {
      "saved_object": {
        "id": "1758f040-65cc-11ed-867e-318c4bd4f121",
        "type": "alert"
      },
      "session_id": "abcZLiBEtrFLn7BQNvtEcGgsZ59G/2ZfUd2qiT93aRY=",
      "space_id": "default"
    },

The problem is that the kibana.* fields from the source event are not added to the generated alerts, because we strip all of them out:

https://github.com/elastic/kibana/blob/a9c8e8f33b30547c815fb598943c354fcd4ee6c2/x-pack/plugins/security_solution/server/lib/detection_engine/rule_types/factories/utils/filter_source.ts#L11-L35

This was introduced in https://github.com/elastic/kibana/pull/148923 to fix a bug.

Maybe we can do it a bit smarter. Maybe we could remove only the kibana.alert.* fields we generate, and copy the rest of the kibana.* fields to the alert object.

elasticmachine commented 4 months ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 4 months ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 4 months ago

Pinging @elastic/security-detection-engine (Team:Detection Engine)

yctercero commented 3 months ago

We've had a number of requests to expand the mapped fields on the alerts index and I think this goes along with that. Trying to gather these issues to do some research around what it would take and any concerns that may exist with such an effort - https://github.com/elastic/kibana/issues/184694