GHPRI issue summary doesn't know who current user is

[lramos15]

Testing #6321

@githubpr Summarize my work items for https://github.com/microsoft/vscode-copilot/issues/8836

Doesn't provide a better summary than just asking for a generic summary. I would like it to give me a contextual response and know that my GH tag is lramos15

[aiday-mar]

I have started looking into how to fix that. To do that I would like to do the following:

• include the current user's login in the summarization tool prompt
• include the user's actual request in the summarization tool prompt

The reason I'd like to include the user's actual request is because I'd like the summarization tool to take into account the details of how summarization is done, that is in Logan's case, the summary should be only for the current user.

I added the user's request into our invocationOptions object of type LanguageModelToolInvocationOptions<object> that is passed in when calling vscode.lm.invokeTool. Meaning I did the following:

const invocationOptions: ToolInvocationOptions<any> = {
    parameters,
    requestPrompt: request.prompt,
    toolInvocationToken: request.toolInvocationToken
};
toolCalls.push({
    call: part,
    result: vscode.lm.invokeTool(tool.name, invocationOptions, token),
    tool
});

where ToolInvocationOptions extends from vscode.LanguageModelToolInvocationOptions. When testing I noticed that when calling invoke, the options object does not contain the field requestPrompt. I then looked on the definition of the method invokeTool on the VS Code side. I noticed that in the file extHostLanguageModelTools.ts on line 53, we essentially destructure the options object and call $invokeTool with only the parameters, the tokenizationOptions and the toolInvocationToken.

I see two ways to fix this:

• Append directly the request prompt to the parameters object which is returned in the model result stream, and use it in the tool invocation
• Perhaps we could generally add a new optional parameter called context which would be defined by the developer and which we could use to pass in additional data, into the tool invocation

Thoughts @roblourens ?

[roblourens]

There's a spectrum of options. For a tool that you register with registerTool, it's best to be able to have the LLM call the tool without any special handling of a particular tool. So then the "proper" thing to do IMO would be to figure out how to map that part of a request to a tool parameter that the LLM can fill in. I don't know what that would look like for your tool, maybe something like "filter": "currentUser" or something. If you need to do some other processing, you can also do "query": "Summarize my work items" and then set the description of "query" to something that will convince the LLM to fill it out correctly.

Then, it's sort of cheating, but you can also have a "query" that you just set manually in your code when this tool is called.

Besides that, I think it's more appropriate to have more flexibility like this if you don't use registerTool/invokeTool but rather just have a made-up tool that only exists in your code- then you can have this disconnect between the tool that the LLM calls, and you run whatever code you want internally to get a result that you report to the LLM.

Happy to chat about it more if this is confusing.

[aiday-mar]

Thanks for the answer. I will think more about that.