Users should be able to modify conversation history that is sent to the API.

[Olyray]

Validations

• [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that requests the same enhancement

Problem

The problem is that of cost. Apparently, the AI is stateless, meaning that each query that is sent to the AI is sent along with the pre-existing context in the chat.

What this means is that the more a user interacts with the AI, the more tokens are consumed by the API as the context has to be sent each time.

Solution

Since the AI is stateless, the solution would be to allow users modify the context. Users should be able to delete messages from the context history. That way, only relevant context would be sent to the API. The alternative of starting a new chat every time is bad user experience to be honest. This is because there might be some context that you want to include in the chat, but starting afresh means you'll lose the context.

If users can delete irrelevant messages from the context, then they can delete the irrelevant messages and leave the relevant ones. Making it possible for them to better manage the number of tokens consumed by the API.

[joshvera]

Opened a PR to address this with a Delete Message button at https://github.com/continuedev/continue/pull/1790

[Olyray]

Opened a PR to address this with a Delete Message button at #1790

Oh. Wow. Nice!