Paragraph streamed at once in a non-smooth manner + references view is open and then closes as response is streamed in

microsoft / vscode

Visual Studio Code

https://code.visualstudio.com

MIT License

164.42k stars 29.33k forks source link

Paragraph streamed at once in a non-smooth manner + references view is open and then closes as response is streamed in #217907

Closed aiday-mar closed 4 months ago

aiday-mar commented 4 months ago

Testing #217639

In the following example where I ran the command @workspace /explain, I noticed two things which I found a bit unexpected.

After the 1st and the 2nd point were streamed in, there was a pause and the 3rd point was streamed in directly in a block. This looked a bit sudden. I am wondering if this is something we control and if we can make streaming more smooth? The same happened for the 4th point.
Initially the references view was open and then it closed before streaming in the response. I guess this is maybe intentional so that the references view does not take up too much space in the panel, it just looked at bit sudden too.

https://github.com/microsoft/vscode/assets/61460952/84098707-baa3-4ead-b0f7-92fec709f6d7

The output in the Window is appended here in a word file.

testing.docx

roblourens commented 4 months ago

This is probably related to the links and the linkification process. Not sure whether it can be smoothed out without delaying the rendering more but I want to investigate some more.

roblourens commented 4 months ago

I think the linkification might introduce pauses sometimes, maybe when statting the file before sending the link. But also, it seems like the stream from the extension is a little 'bursty', it seems to slow down or speed up. Not sure whether this is from CAPI or from what we do, it's probably worth investigating more.

It's also probably worth trying to get on to the anchor chat API. We could do that validation async on the frontend, if that's why it's slow.

roblourens commented 4 months ago

An easy way to see the real stream is to edit getProgressiveRenderRate in ChatListRenderer.ts and add return 10000 at the top.

mjbvz commented 4 months ago

Problem appears to the the getDefinitions call here: https://github.com/microsoft/vscode-copilot/blob/90c4864f48cee454e951051e8f025c9aa4c4e980/src/extension/linkify/vscode-node/inlineCodeSymbolLinkifier.ts#L105

Instead we can likely defer this until the link is clicked. If a definition is not found, we can open the matching symbol instead

mjbvz commented 4 months ago

Should be fixed upstream