elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.82k stars 8.2k forks source link

[Security GenAI] When indices referenced in KB index entries are deleted from OUTSIDE the AI Assistant KB UI, there is not indication to the user #197156

Open peluja1012 opened 2 hours ago

peluja1012 commented 2 hours ago

Describe the bug: When Knowledge indices referenced by KB index entries are deleted from OUTSIDE the AI Assistant knowledge base UI, they remain visible in the list of knowledge sources, even though the source index no longer exists. When users ask a question related to these KB entries, the AI Assistant will return either an error or a message indicating that it doesn't have the knowledge

To improve this experience:

  1. We should display some sort of indication (warning) in the list of KB sources that shows that index has been deleted.
  2. The assistant should respond with a consistent message. Currently, sometimes we get raw tool responses, sometimes we get a nicer answer from the LLM (see screenshots below)

Kibana/Elasticsearch Stack version: 8.16 BC1

Steps to reproduce:

  1. Create an index KB entry in Stack Management -> AI Assistants -> Knowledge Base.
  2. Delete the index referenced by the KB entry via Dev Tools
  3. Navigate to AI Assistant and ask a question related to this KB entry.
  4. Notice that the AI Assistant returns an error message or a message indicating that it doesn't have related knowledge.

Screenshots: Image Image

elasticmachine commented 2 hours ago

Pinging @elastic/security-solution (Team: SecuritySolution)