microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
328 stars 31 forks source link

Add a feature to edit user messages #15

Closed Chriss4123 closed 10 months ago

Chriss4123 commented 1 year ago

Currently, you are unable to edit user messages. This is very painful as if you make a small mistake in your question, the LLM performs worse when a follow up questions is given compared to if you were to put the correct prompt the first time around.

GreenWizard2015 commented 1 year ago

Yes, this feature is extremely important. As I mentioned in the article, it is very important to maintain a clean/small context. It would be great if, upon pressing "up", the message could be edited instead of being duplicated. An ideal situation would be the ability to work with the chat like with documents and edit any part of it. At the same time, for example, you can remove insignificant parts of the AI's response or correct errors. Such collaboration works great when working on code with Copilot. I have experimented a bit with this approach through the OpenAI API, and it is very useful. The possibility to work with the chat like with documents also enhances accessibility/usability. The ability to use only the keyboard to interact with the chat is a significant advantage for me.

Chriss4123 commented 1 year ago

Yes, this feature is extremely important. As I mentioned in the article, it is very important to maintain a clean/small context. It would be great if, upon pressing "up", the message could be edited instead of being duplicated. An ideal situation would be the ability to work with the chat like with documents and edit any part of it. At the same time, for example, you can remove insignificant parts of the AI's response or correct errors. Such collaboration works great when working on code with Copilot. I have experimented a bit with this approach through the OpenAI API, and it is very useful. The possibility to work with the chat like with documents also enhances accessibility/usability. The ability to use only the keyboard to interact with the chat is a significant advantage for me.

I definitely agree. I would've also added editing the system message and the AI-generated responses in the feature request, however this would likely not get added because it would increase the risk of the LLM being used for questions other than programming by starting off the conversation something like:

System: "You are going to have an open-ended-discussion with this user" User: "Can I ask you anything?" Assistant: "Sure, you can ask me about anything. It does not have to be programming related." User: non-programming question

But apart from that, people who are using tools like these in order to help them write better quality code would really benefit from being able to edit and manipulate previous messages in the conversation, as it is undoubtedly one of the best ways to extract higher-quality answers from a LLM like GPT-3, 3.5 or 4.

I really enjoyed reading the mentioned article you authored. Keep it up!

GreenWizard2015 commented 1 year ago

...however this would likely not get added because it would increase the risk of the LLM being used for questions other than programming...

This makes sense, but the question is quite complex, and I see many nuances. In brief, it is possible to bypass security measures in order to misuse the system, but also these security measures can harm legitimate users. For example, writing documentation and a README is a valid task, but it is not directly related to programming/code.

But apart from that, people who are using tools like these in order to help them write better quality code would really benefit from being able to edit and manipulate previous messages in the conversation, as it is undoubtedly one of the best ways to extract higher-quality answers from a LLM like GPT-3, 3.5 or 4.

The ability to edit the entire dialogue can indeed create risks. However, the editing system used in the web version of ChatGPT has, in my opinion, proven its security. If the chain of responses following the edited one is deleted, there should be no security issues because technically it's just N-1 messages and one new message that requires verification.

The ability to navigate through the dialogue using the keyboard, but without the ability to edit, also poses no risks. Moreover, I believe it will greatly enhance UX.

Chriss4123 commented 1 year ago

The ability to edit the entire dialogue can indeed create risks. However, the editing system used in the web version of ChatGPT has, in my opinion, proven its security. If the chain of responses following the edited one is deleted, there should be no security issues because technically it's just N-1 messages and one new message that requires verification.

100%. ChatGPT has definitely proven itself with their system of message editing where you are only allowed to edit user messages and after you do edit a message, the messages list is truncated to the edited message meaning you can't really abuse the model in that way. I'd be happy if that kind of system was implemented and even more so if the ability to edit AI responses is added, we'll just have to wait and see.

Regarding the keyboard shortcuts, on point as well. As a heavy vim user using keyboard shortcuts to improve my efficiency to the maximum is second nature to me. A few crucial keybinds I'd like to see is one for editing the previous message, regenerating the response and clearing the chat however a lot more can be added to make this useful tool even more efficient.

roblourens commented 1 year ago

I've added a feature to let you delete messages, which you can then replace. I think this is a useful lightweight approach to the problem, does it help you?

julian-shalaby commented 1 year ago

Here's an alternative which allows you to do so. It gives much better context management