search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
160.19k stars 28.06k forks source link

Chat tools API #213274

Open roblourens opened 1 month ago

roblourens commented 1 month ago

This is a proposal to enable tool use/function calling from chat participants and other extensions that use language models.

Proposal dts: https://github.com/microsoft/vscode/blob/b1252f75d52bcc576ca58415512343a8413fa260/src/vscode-dts/vscode.proposed.chatTools.d.ts

Sample: https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample Docs: https://code.visualstudio.com/api/extension-guides/chat#variables

Extension authors can subscribe to this issue to get updates about the proposed Tools API.

We are very interested in feedback about how you might use this API.

jdchmiel commented 1 month ago

Forgive me if this is the wrong place - I was curious what the model is looking like for when a chat response contains buttons, which map to commands, which then need to use the ChatResponseStream to insert a response from the participant into the chat window. I have not figured out how to have the command handler gain access to the ChatResponseStream? It is a common use case for the tools I would like to leverage that will be doing something external in a terminal or an API and then presenting a conversational response within the chat window, so I think I am just overlooking the obvious for how chat tools will relay conversational messaging back?

roblourens commented 1 month ago

That's not a case that is supported right now. There's no way to change the chat response after the chat participant handler is complete.

jdchmiel commented 1 month ago

Thank you for your response. Do you have any ideas on alternate ways to accomplish invocation of a command ( or local code in any abstraction) and returning the results to the chat? Would additional use cases help? Like another very simple one might be "why did the build fail" and I would like to parse a CICD platform and respond with the failure reasoning in chat, as a link to the build perhaps.

roblourens commented 1 month ago

You can call executeCommand('workbench.action.chat.open', 'query') to invoke your participant again, and continue the conversation. That API is a little primitive and I'd like to have a more natural mechanism for this. I opened this issue to track it https://github.com/microsoft/vscode/issues/213827. You can reply there if you want to discuss further