[Request] Retrieval Augemented Generation (RAG) for Alerts

[dhru42]

Description

What: We're implementing Retrieval Augmented Generation for Alerts Why: Currently, the AI Assistant provides one alert or event at a time as context to the LLM, but has no way to answer questions about multiple alerts in the alerts index. With this functionality, the assistant can provide the riskiest, latest alerts as context to the LLM, with data anonymized per a user’s settings. Example of new questions users can ask: _- How many alerts are currently open?

• Which alerts should I look at first?
• Did we have any alerts with suspicious activity on Windows machines?_

How:

• Enables multiple alerts to be passed by the server as context to the LLM
• Applies the user's anonymization settings to those alerts
  • Only fields allowed by the user will be sent as context to the LLM
  • Users may enable or disable anonymization for specific fields (via settings)
  • Click the conversation's Show anonymized toggle to see the anonymized values sent to / received from the LLM:

Settings

This feature is enabled and configured via the Knowledge Base > Alerts settings in the screenshot below:

• The Alerts toggle enables or disables the feature
• The slider has a range of 10 - 100 alerts (default: 20)

When the setting above is enabled, up to n alerts (as determined by the slider) that meet the following criteria will be returned:

• the kibana.alert.workflow_status must be open
• the alert must have been generated in the last 24 hours
• the alert must NOT be a kibana.alert.building_block_type alert
• the n alerts are ordered by kibana.alert.risk_score, to prioritize the riskiest alerts

Background & resources

• PRs: https://github.com/elastic/kibana/pull/172542
• Issues/metas: https://github.com/orgs/elastic/projects/1301/views/3?pane=issue&itemId=41046323
• Point of contact: @dhru42 @jamesspi @andrew-goldstein @spong
• Test environments:

Which documentation set does this change impact?

ESS and serverless

ESS release

8.12

Serverless release

tbd

Feature differences

N/A

API docs impact

n/a

Prerequisites, privileges, feature flags

• Feature Flag: Yes

Feature flag To use this feature:

Add the assistantRagOnAlerts feature flag to the xpack.securitySolution.enableExperimental setting in config/kibana.yml (or config/kibana.dev.yml in local development environments), per the example below: xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts'] Enable the Alerts toggle in the Assistant's Knowledge Base settings, per the screenshot below:

[dhru42]

@benironside @joepeeples please let us know if you want to get together for a live demo of this feature. Given the new guidance regarding FF's, we'd love the ability to release this feature as enabled behind a feature flag if docs are available. Please let us know if that will be possible, and we're happy to help out in any way we can.

cc: @jamesspi @andrew-goldstein @spong

[andrew-goldstein]

Thanks for documenting this feature!

We made a few small changes in this PR: https://github.com/elastic/kibana/pull/173121

• acknowledged alerts are now included as context (in addition to open) alerts
• The following text in settings was updated:

• The text Ask questions about the (n) latest and riskiest open and acknowledged alerts in your environment. was updated to include acknowledged alerts
• The additional text Select fewer alerts if the model's maximum context length is frequently exceeded. was added because users may encounter the following error when they exceed the model's maximum context length:

We're happy to improve the settings text with your feedback if you're willing to provide it.