AI Chat service refactor to support multiple-conversation, persistence, and standalone page

[petemill]

As of now, most AI Chat functionality is handled by a TabHelper, reflecting the single use-case focused around an active Tab, via the sidebar. Those conversations are not persisted and not accessible outside of the Sidebar for the specific Tab.

A refactor will support:

• Keeping track of multiple conversations in one place (the service)
• Persisting conversations to a data store
• Allowing non-persistence mode for a Profile (e.g. Incognito)
• Cleanly making model and engine choices, building different prompt styles for each, and sending the prompt either locally or to an API with different params
• Starting new conversations from the UI
• Regenerating completion responses and keeping those alternatives

---

AIChatService (Profile KeyedService)

Manages conversations for the profile. Stores all conversations in database and manages retrieval.

• Functions

  • GetAllConversations() - Fetches profile’s Conversation metadata (id, title / first message, related Url), not conversation messages
  • _GetConversation(id):aichat::Conversation - Fetches conversation messages and data
  • GetConversationsWithAssociatedPageContent(page: AssociatedPageContent) - Retrieves 0..n Conversations that are attached to the page content (Tab/WebContents for Desktop/Android)
  • ClearAllConversations - Remove all conversations when profile data gets deleted or reset
  • _NewConversation(webcontents): Doesn’t get persisted until there’s an entry
    • _web_contents _is not-null if the conversation is associated with a Tab (via sidebar)
  • DeleteConversation(id)
  • IsConversationWebContentsAlive(id)
    • Returns true if there is an open WebContents for the conversation, and the page contents is retrievable (or in-memory).
    • Returns false if the conversation was related to a WebContents, but the WebContents is now destroyed.
  • SetConversationActive(id)
    • UI tells service when a conversation is open or closed by the user, which is passed to ai_chat::Conversation
    • Enables the conversation to decide when to do automatic actions, such as question retrieval or content distilling.

• Events

  • OnConversationAdded
  • OnConversationRemoved

• Persistence

  • GetAllConversations, GetConversation(id) retrieves data
  • Conversation::OnConversationEntriesUpdated updates data
  • DeleteConversations, ClearAllConversations deletes data

• Owns

  • AIChatAPI (i.e. APIRequestHelper)
  • All active ai_chat::Conversation instances

AssociatedContentDelegate (on desktop implemented by AIChatTabHelper)

Supplements a conversation with page content

• Functions
  • GetPageContent: string, PageContentType
  • GetAssociatedConversation: ai_chat::Conversation
  • SetAssociatedConversation(conversation)

ai_chat::Conversation

In-memory store for a conversation and its data. Instances are owned by AIChatService. \ Builds prompts for new messages based on conversation history and metadata.

• Functions
  • GetConversationEntries()
  • GetSuggestedQuesions()
  • SubmitHumanInput(text)
    • Builds prompt and makes API call.
    • Successful response is added to conversation and Service is notified so that it can persist.t
  • RegenerateEntry(id)
    • Same as SubmitHumanInput except the query is re-sent and an alternative response is added to the entry.
  • SetConversationActive(is_active)
  • {Set,Get}AssociatedContentDelegate
• Events
  • OnConversationEntriesUpdated
  • OnSuggestedQuestionsChanged
  • OnPageContentUpdated
  • OnConversationError
• State
  • associated_content_delegate: AssociatedContentDelegate. When this is not null (or not destroyed), this object will use web_contents data to affect the conversation via the AIChatTabHelper.
  • is_active: bool
  • entries: ai_chat::ConversationEntry[]

ai_chat::ConversationEntry

struct

• CharacterType: HUMAN|ASSISTANT
• ConversationVisibility: PUBLIC|INTERNAL // Not used anymore, perhaps we remove
• Texts: string[] // An array in order to support alternative generations
• Date
• ID

ai_chat::engine_consuming

A set of functions which build and use various prompts based on model and data, makes model decisions, and sends requests to either local LLM or remote API. Might use other files for engine/model-specific constants and algorithms

• Functions
  • SubmitHumanInput(conversation, text): ConversationEntry
    • Prompt is based on conversation’s history and context, e.g. if it has associated page content or not. Human input and page content is sanitized according to the prompt’s components.
  • RegenerateAssistantResponse(conversation, entry_id): text
  • SubmitSummarizationInput(conversation): ConversationEntry
  • GenerateQuestionSuggestions(conversation): Questions[]

---

UI Flow

The mojom between WebUI C++ and WebUI JS will change to provide functions for starting new conversations, getting the conversations list metadata, and providing the entries and other state properties of the active conversation. It will also provide events for both when the active conversation state changes and when the list of all conversations’ metadata changes.

Sidebar

The WebUI c++ knows it is open as a sidebar and has a target Tab WebContents. The UI can then be provided with a specific Conversation to be open at via AIChatTabHelper::GetAssociatedConversation, which will return an existing or new Conversation. The UI will also allow a different conversation to be chosen and made active.

Standalone

The page will open with no conversation selected. A list of all conversations will be displayed as well as a prompt to start a new conversation. Any conversation can be chosen and made active, and new conversations can be created.

Incognito

Incognito and regular profiles will be treated as independent. Incognito version will not use any persistence.

Android

The same WebUI will be used on Android

---

Storage

We will use SQLite to store all conversations from the regular profile. No conversations from incognito profile will be persisted.

• Conversations
  • Metadata (title, datetime, related URL)
  • All ConversationEntry instances
  • Page text and type, if applicable

    
    ### Tasks

• [x] Prototype storage class and adjust spec accordingly
• [ ] https://github.com/brave/brave-browser/issues/31821
• [x] extract TabHelper Conversation and ConversationEntry logic and state to Conversation class
• [x] Build AIChatService and persistance
• [x] Tests

[yrliou]

Just a quick note, I don't think we should use web_contents directly in Conversation(Driver), we shouldn't use something from content layer here, instead, I think we might want a delegate maybe named PageContextDelegate which AIChatTabHelper would implement.

[yrliou]

Things currently owned by conversation_driver, like ai_chatmetrics(not owned instance), credentialmanager, feedbackapi, should be owned by AIChatService instead if it doesn't need to be one instance for each conversation. Needed things can be passed in the service factory. And Conversation(Driver) can have a raw_ptr if needed for those which will be passed when AIChatService create the conversation.

We can also create and own instances of EngineConsumerClaude, EngineConsumerLlama in AIChatService, and take model parameter for functions which submit queries.

We should also remove the inheritance between tab_helper and conversation_driver, creation of tab_helper will need less parameters since most of things are needed by conversation_driver and not the tab_helper.

[petemill]

Things currently owned by conversation_driver, like ai_chatmetrics(not owned instance), credentialmanager, feedbackapi, should be owned by AIChatService instead

Absolutely

We should also remove the inheritance between tab_helper and conversation_driver, creation of tab_helper will need less parameters since most of things are needed by conversation_driver and not the tab_helper.

Yes that was the plan I had too. I thought it was explicitly mentioned above but maybe an edit wasn't submitted a while ago, or maybe it was in a DM conversation with someone when iOS was started and ConversationDriver was extracted from the original AIChatTabHelper, but either way the above was actually designed before ConversationDriver subclassing AIChatTabHelper was implemented for iOS. So the implication was that Conversation and the AIChatTabHelper were separate classes with an optional association. Content association should be via delegate so that standalone conversations will be nicer.