Explore Python IntelliSense in Copilot chat code blocks

[mjbvz]

This is an open ended, exploratory item

VS Code recently enabled intellisense in copilot chat code blocks. Although this is mostly implemented in a language agnostic way, language extensions may need to make some changes to get the best support

For this exploration, I'd like the python team's help testing out IntelliSense in copilot chat code blocks. I've provided a list of ideas on what to test below, however feel free to build on it. I've gone through them for JS/TS and found them to be a good starting point. You can also file feature request against VS Code for new feature ideas

Areas to test

• The Python extension should see TextDocuments for each chat code block

  There will actually be two docs per code block, one with the scheme vscode-chat-code-block: and one with the scheme vscode-copilot-chat-code-block:. vscode-chat-code-block is the scheme used by VS Code core to provide basic intellisense (such as expand selection. vscode-copilot-chat-code-block is the scheme used by copilot to provide go to definition support. This is needed because copilot registers itself as the go to def provider for vscode-chat-code-block

• Basic syntax language features — such bracket matching and Smart Selection — should always work

• You should never see errors reported for code in chat code blocks

  Try asking copilot to generate invalid code to confirm this

• Test hovers and go to definition within a single code block

  For example if you ask for binary search in python, there should be a single code block that has these IntelliSense features

  • Make sure these features work for global symbols
  • Also test IntelliSense for symbols defined within the block
  • Check that any special python features in hovers (such as links) work as expected
  • Make sure the code block IntelliSense observes any relevant workspace/project settings

• Test cross code block IntelliSense

  Ask a questions such as binary search in python and follow up by asking write some tests for it. The second code block should use the binary search function defined in the first. Inside of the second code block you should be able to hover on it to see documentation and use go to definition on it

• Test IntelliSense for workspace symbols

  This feature is mostly handled by copilot but still worth testing. For it, as a @workspace question to find code in the workspace. For example, @workspace where's the code for binary search

  • Make sure can use hover, go to definition, go to implementation, and go to type definition for symbols that come from the workspace
  • Make sure you can still use these features for globals and locals too

• Make sure that code in code blocks is never picked up as a reference

  If a code block uses a global symbol for example, when you run find all references on that symbol in a normal workspace file, you should not see the use in the code block listed

• Make sure symbols code blocks never shows up in workspace symbol search

• Explore any python specific features/behavior of these IntelliSense features inside these chat code blocks

• Think about if there's any additional IntelliSense features that would help users understand code inside of chat code blocks

[rchiodo]

I'm wondering if our import resolution is going to work. The schema will make us think the file is not on the local file system, so we won't search the workspace. We might have to special case the schema here.

[heejaechang]

default workspace might work with stdlib with any random url in async server. but that said, feels like it might be better to just create new workspace that deals with copilot specifically? like copilotWorkspace ? and we enable or disable certain features there?

[luabud]

next week I'll try to look more deeply into this, since right now we only support vscode-chat-code-block and I was struggling to get a Pylance build with vscode-copilot-chat-code-block and I'm not sure why.

But so far I noticed that when Copilot uses one of the workspace files as reference, go to def and hover works as expected. With reference:

No reference (nothing shows when hovering models)

Also noticed that go to definition on .models takes to the Django built-in models module, not my models.py file. But the classes go to the models.py as expected:

Errors are not being displayed indeed, although a few diagnostics such as unaccessed symbol or unreachable code are displayed (note that the "experimental pylance" string is a known bug that has just recently been fixed):

Apart from that, everything else worked well: bracket matching, smart selection, go to def on global variables, response results not showing on symbol search

[luabud]

@mjbvz is there anything special we should do to support this for multi-root workspaces? or should it be working out of the box?

[mjbvz]

Chat searches through all workspaces so a generated code block may come from files across workspaces. You'll need to make sure Python can correctly handle this when resolving symbols in the code block text document

Beyond that though the feature should mostly just work for multi root workspaces

[heejaechang]

@mjbvz I think it is very hard for us to support this in its current form. For example, we don't even know when to clear states we create and hold for these random LSP requests. Also, a generated code block may come from files across workspaces, I think we won't be able to support this in any stable way.

If we want this to work, vscode should send something like onDidChangeWorkspaceFolders (workspace/didChangeWorkspaceFolders) for the code block and remove them when you are done with the code block.

If you want to group some code blocks (virtual files), put them in the same workspace (under the same root URI) so the code blocks can see each other. Otherwise, put them in different workspaces (under different root uri).

When they are done, by removing the workspace, you let us know when we can clear states.

As for the referenced files for the code block, I am not 100% sure how you want us to handle it. Since resolving imports/symbols requires knowing where the root is to properly handle them, regardless we search all workspace/files (that could be expensive), without all necessary information, we won't be able to do it reliably. we could do some best effort stuff with heuristic, but that will always feel work randomly.

if the URL is not opaque, you can add those referenced files through fileopen with a specific URI (such as vscode-xxx://rootUri-for-code-block-workspace?file=/fullpath/to/file.py) so we can map the virtual file you added to the actual file on disk. and from there, we can figure out root of workspace for the file.py and how to resolve it correctly.

open to other idea, but I am not 100% sure how we are supposed to figure out what user file/module/symbol the import statement is referring to from random text. we can do that for stdlib, typeshed, site-packages ones assuming they are using default ones.

or my expectation is too high? when what we are looking for is any kind of thing sometime work?

[luabud]

I'm thinking this would be a good start?

we can do that for stdlib, typeshed, site-packages ones assuming they are using default ones.

We could potentially also support this for 3rd party packages if they're installed in the workspace (e.g. supporting go to def on test below)

To @heejaechang's point on how can Pylance know when to "let go" of the code blocks, the suggestion below sounded like a good idea?

If we want this to work, vscode should send something like onDidChangeWorkspaceFolders (workspace/didChangeWorkspaceFolders) for the code block and remove them when you are done with the code block.

@mjbvz would that make sense as a feature request to VS Code core?

[heejaechang]

@luabud ya, we can support other import roots that we can get from settings such as python interpreter or extra paths and etc that we would normally get for default workspace. we could support user files by adding all regular workspace roots in extra path, but that will only work if generated code has import statement from the workspace root like from helloworld.models

[mjbvz]

@luabud Restricting this to stdlib and 3rd party packages sounds reasonable to me. Resolving local files is something we should investigate longer term but is much more complicated

Your extension should not worrying about manage document lifecycle manually. Instead VS Code will create and destroy the relevant vscode-chat-code-block scheme text documents for you (you will see this behavior in the next insiders build). The LSP will synchronize these the same way as it does with other in-memory text documents

I don't think using workspaces makes sense. However chat code blocks uri do use the conversation id as their authority so they can be grouped by conversation:

vscode-chat-code-block://conversation-id/code-block-id

[heejaechang]

@mjbvz sure, we can use that to create implicit workspace ourselves. that said, you wanted for us to refer symbols defined in other code block in other code block without explicit import like notebook cells? is that right? in that case, we could internally just treat each code block like cells and each coversation-id as new notebook of its own workspace (I guess and the virtual notebook has same python interpreter as what user selected for vscode?)