[Security Assistant] - Conversation pagination, default conversations, and new conversation creation

[stephmilovic]

An investigation into system prompts has uncovered several issues related to conversation management, UI behavior, and system prompt assignments, leading to persistent errors and inconsistent functionality.

Main Issues:

1. Incomplete Conversation Retrieval:

   • The client is not sending the per_page parameter, resulting in a limited number of conversations being retrieved from the server.
   • Consideration: Retrieving the entire list of conversations may not be efficient; perhaps the system should use pagination or another strategy to fetch only necessary conversations dynamically.

2. Memory Loading Assumptions:

   • The system assumes all conversations are preloaded into memory, which leads to:
     • "Welcome" Toaster Errors: These errors occur because the system tries to reference the "Welcome" conversation even when it doesn’t exist, leading to persistent error notifications.
     • Combo Box Limitation: The "Default Conversations" combo box in the System Prompts settings is unable to access all conversations, likely due to the incomplete retrieval.

3. Default Conversations Behavior:

   • Issue: Default conversations are handled differently than regular conversations, creating inconsistency in behavior.
   • Suggestions:
     • Default conversations should be created immediately and treated like regular conversations to avoid these discrepancies.
     • There’s a possibility that default conversations could be removed altogether, as discussed by the design team.

4. New Chat Creation Issues:

   • A new chat is created when the "New Chat" button is pressed, rather than when the first message is sent
   • The same happens for Attack discovery. We should wait until the message is sent before creating the conversation

5. System Prompt Assignment Bugs:

   • Bugs can occur when system prompts are assigned or unassigned for all conversations, leading to unexpected failures or behavior.

Expected Behavior:

• The system should efficiently retrieve conversations without needing to load all at once.
• New chats should be created upon the first message being sent, not at the button click.

[stephmilovic]

This was discussed in the team meeting on 9/12/24 and we decided in the short term (8.16) we will patch the bug by fixing the client per_page query parameter to correctly fetch "all" the conversations (10,000). We should only fetch the messages portion of the conversation when the conversation is active in the UI.