Elastic Security Assistant overview (docs for 8.8.1)

[jmikell821]

Related issues/PRs:

• Epic: https://github.com/elastic/security-team/issues/6775
• Kibana PR: https://github.com/elastic/kibana/pull/156933
• Another one: https://github.com/elastic/kibana/pull/159054

User Story

Iris, Elastic Security's AI assistant, integrates generative AI and large language models (LLMs) into the workflows of Elastic Security users. Incorporating Generative AI into Elastic Security vastly improves our threat response capability, productivity, and performance, empowering our security teams. The addition saves time on routine tasks, decreases threat resolution time, and heightens workflow efficiency and outcomes.

As a Security Analyst... (note that some of these features are not 100% mature yet)

• I want to be able to invoke the Security Assistant with prebuilt contexts so that I can better understand alerts, timeline events, and prebuilt detection rules. This will save me time and allow me to focus on threat mitigation rather than interpretation.
• I want the Security Assistant to safeguard our company’s sensitive data by anonymizing it before any external requests and subsequently recontextualizing the response. This ensures data security without compromising the benefits of external AI services.
• I want the AI to be able to save chat results to timeline notes and/or cases within Elastic Security. This will create a record of AI responses, allowing us to refer back to them for future analysis, audits, or to provide context for new team members.
• I want the AI to present a token count to me so that I can have an indication of usage and manage our resources more effectively.

Prerequisites

This functionality has been introduced behind the assistantEnabled feature flag. Customers need to add this configuration to their kibana.yml or Kibana Cloud User Settings configuration to enable:

xpack.securitySolution.enableExperimental: ['assistantEnabled']

Notes

• We need to include the following:

  • How to enable the overall feature (feature flag above)
  • How to enable the shortcut (Cmd + ; for macOS, Ctrl + ; for Windows)
  • How to add the Generative AI connector
  • Three UI buttons that enable you to copy to clipboard, etc.
  • Example of how to use the chat assistant with Alert summary
  • Call out caveats, such as that customers need to exercise caution when entering prompts, as data is NOT anonymous. Data might also contain errors
  • Brief reference to the quick prompts (they're located at the bottom of the chat window and they're useful for quickly entering commands).

• The name "Iris" might not make it into the initial MVP for 8.8.1; use "Security Assistant" until the official name is established.

• This will be backported to 8.8.1 but not 8.8.0. Add note to 8.8 docs (since we don't publish patch-level docs, just minor)?

• Licensed for Enterprise subscription.

• AI model versioning: For OpenAI, the api will end up using the gpt-3.5-turbo model. For Azure, any that they setup.

• Caveat verbiage (first draft):

  This is an initial release of the Elastic Security assistant. While it's designed to enhance your analysis with smart dialogues, its capabilities are still developing. Users should leverage it sensibly as the reliability of its responses might vary. Your insights, patience, and feedback help us calibrate this feature for optimal use. Always cross-verify any returned advice for accurate cyber security threat detection and response, insights and query generation.

• Also need to warn users that data they enter is not anonymized; anything they put into Security Assistant will go to the third-party AI provider.

[nastasha-solomon]

Some additional details I noted down from today's meeting with James:

• It's very likely that alert documents will contain sensitive data or PID so definitely want to make users aware that their data isn't being anonymized when they're sending it to third-party language model.
• Conversations and outputs are stored in a users browser state. This might mean that, if users open a new browser window/session, they'll be forced to start a brand new convo with the assistant.
• About conversations:
  • Users don't need to navigate to the relevant page (e.g., Timeline, Alerts, Data Quality Dashboard) to start a conversation about a specific topic.
  • Conversations are helpful for keeping threads organized and segmented.
  • Users can use the "Welcome" conversation for general topics and questions that don't fall into any of the existing conversation categories.
• About the feature flag xpack.securitySolution.enableExperimental: ['assistantEnabled']:
  • The feature flag for enabling the assistant also enables the Generative AI connector.
  • If users delete the feature flag from their kibana.yml after creating and adding a Generative AI connector, the connector will appear greyed out on the Connectors page.
• Users might run into service timeout errors. The copy button is helpful for re-entering and running commands/inputs if this happens.