microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
295 stars 28 forks source link

`@workspace` can't find anything through my code base #529

Open alessandro-newzoo opened 8 months ago

alessandro-newzoo commented 8 months ago

Hello, I was really excited about workspace codebase awareness, but I sadly can't make it work.

As you can see here, it doesn't find what I'm looking for, even though it's obvious from the search on the left that this var is defined somewhere. I also don't understand the logic it's using to find the references. I've asked it to look into php files, and it looks into composer.json and the workspace file? How did it pick that one php file specifically, since I have dozens of other php files?

image

I then ran it a second time with different wording, completely different and random results:

image

In case it's relevant, what you see is a multi-root workspace in a GH Codespace, on VS Code Desktop. The multi-root workspace has two folders: THEME and DDEV.

Update: same issue on a local single-root repo: https://github.com/microsoft/vscode-copilot-release/issues/529#issuecomment-1844867361

kcaswick commented 8 months ago

I'm having similar issues in a single root, TypeScript JSX workspace. It only seems to be including selected text as context, even when I explicitly mention other files.

REDACTED: @workspace Please compare the functions in ResourceDetails.fieldGroup.tsx and ResourceRequest.fieldGroup.tsx

GitHub Copilot: Used 1 reference: Untitled-1:43-65 I'm sorry, I can't answer that question with what I currently know about your workspace. I don't have access to your ResourceDetails.fieldGroup.tsx and ResourceRequest.fieldGroup.tsx files to compare the functions.

I also tried copying the relative path from an open tab, then trying to mention it while in another tab:

REDACTED: @workspace /explain src\packages\relative path to file\ResourceDetails.fieldGroup.tsx

GitHub Copilot: Used 1 reference: Untitled-1:43-65 I'm sorry for any confusion, but as an AI, I don't have access to your local or remote files. Therefore, I can't directly analyze the file src\packages\relative path to file\ResourceDetails.fieldGroup.tsx that you mentioned.

However, I can certainly help you create a new component based on the general patterns used in React and TypeScript. If you could provide some details about what's in ResourceDetails.fieldGroup.tsx, such as the structure of the component, the props it takes, and how it handles state and events, I could provide a more accurate example.

jhaisley commented 7 months ago

I'm experiencing a similar issue, @workspace seems to only reference a single open file instead of the workspace, excerpts below are from the code base for an open-source project.

@workspace tell me about data.json

GitHub Copilot
used @workspace
Used 1 reference readme.md:1-58

The [data.json](vscode-file://vscode-app/c:/Users/Jordan%20Haisley/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.html) file is not mentioned in the active selection from your [readme.md](vscode-file://vscode-app/c:/Users/Jordan%20Haisley/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.html). However, based on typical usage, a [data.json](vscode-file://vscode-app/c:/Users/Jordan%20Haisley/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.html) file would be a JSON (JavaScript Object Notation) file used to store data. JSON is a popular data format with a diverse range of applications, such as storing data or as a data exchange format.

Without more specific information about the contents or usage of [data.json](vscode-file://vscode-app/c:/Users/Jordan%20Haisley/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.html) in your project, I can't provide more detailed insights. If you have specific questions about how [data.json](vscode-file://vscode-app/c:/Users/Jordan%20Haisley/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.html) is used in your code, feel free to ask!

Meanwhile if I have data.json open I get a great response, but there seems to be no way to make @workspace work across multiple files.

GitHub Copilot
used @workspace
Used 1 reference data.json:1

GitHub Copilot: The [`data.json`](command:_github.copilot.openRelativePath?%5B%22data.json%22%5D "data.json") file appears to contain a list of JSON objects, each representing an application. Each application object contains various properties such as `AdditionalInfo`, `ApplicationID`, `Application_Source`, `CityName`, `Country`, `CustomQuestions`, `DateReceived`, `EEO_Disability_Status`, `EEO_Ethnicity`, `EEO_Gender`, `EEO_Race`, `EEO_Veteran_Status`, `EducationalExperienceList`, `EmailAddress`, `FirstName`, `JobTitle`, `LastName`, `PersonalReferenceList`, `PhoneNumber`, `ResumeURL`, `SalaryAmount`, `SalaryPeriod`, `StartDate`, `StateName`, and `WorkExperienceList`.

For example, the `AdditionalInfo` property contains a string with information about the applicant's clinical and administrative skills. The `ApplicationID` is a unique identifier for the application. The `CityName` and `Country` properties contain the location of the applicant. The `CustomQuestions` property is a list of questions and their answers. The `EducationalExperienceList` is a list of the applicant's educational experiences, and the `WorkExperienceList` is a list of the applicant's work experiences.

Please note that this is a general interpretation based on the provided excerpt. The actual structure and content of your [`data.json`](command:_github.copilot.openRelativePath?%5B%22data.json%22%5D "data.json") file may vary.
alessandro-newzoo commented 7 months ago

@sbatten @mjbvz will we ever get a response I wonder?

Joroze commented 7 months ago

"I'm sorry, I can't answer that question with what I currently know about your workspace. Please provide more information about your project structure or the files in your workspace."

This occurs for me when my VSCode has multiple workspaces open. It works fine when opening VSCode with a specific project, only.

mjbvz commented 7 months ago

Currently copilot isn't multi-workspace aware. I've updated the issue title to track this

By default it seems like we should search the entire workspace while potentially also providing a way to scope the question to individual workspace root folders

piotrpalek commented 7 months ago

For me this occurs for a single workspace as well, it doesn't really work as advertised and doesn't take into account the workspace context

mjbvz commented 7 months ago

@piotrpalek Please open a separate issue

jhaisley commented 7 months ago

Currently copilot isn't multi-workspace aware. I've updated the issue title to track this

By default it seems like we should search the entire workspace while potentially also providing a way to scope the question to individual workspace root folders

The issue I reported above was a single folder workspace.

You updated the title to exclude the majority of the issues reported under this issue.

mjbvz commented 7 months ago

@jhaisley From original issue text:

In case it's relevant, what you see is a multi-root workspace in a GH Codespace

Open a separate issue

alessandro-newzoo commented 7 months ago

Currently copilot isn't multi-workspace aware. I've updated the issue title to track this

By default it seems like we should search the entire workspace while potentially also providing a way to scope the question to individual workspace root folders

I have removed the 2nd folder from my workspace and I still face the same issue, so I doubt it's got anything to do with my workspace being multi-root.

image image
2023-12-07T08:12:01.754Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:01.754Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T08:12:01.754Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T08:12:08.192Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 6437 ms
2023-12-07T08:12:10.286Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:10.287Z [INFO] [streamChoices] request done: headerRequestId: [55dc51a7-ac4d-453d-bfce-386a92affd4d] model deployment ID: [x74a4b0e7ace1]
2023-12-07T08:12:12.565Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:12.565Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T08:12:12.565Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T08:12:15.839Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 3274 ms
2023-12-07T08:12:17.085Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:17.086Z [INFO] [streamChoices] request done: headerRequestId: [67930983-20f3-4c5e-9960-9d9ba81ffec3] model deployment ID: [x74a4b0e7ace1]
2023-12-07T08:12:17.201Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:17.201Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-12-07T08:12:17.201Z [INFO] [chat fetch] chat model gpt-3.5
2023-12-07T08:12:17.653Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 451 ms
2023-12-07T08:12:17.654Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:17.654Z [INFO] [streamChoices] request done: headerRequestId: [60f325d9-550c-407c-83fb-9061f1459f76] model deployment ID: [x202bdb0a76e7]

I've also just tried with ALL extensions disabled except for Copilot, on a single-root local workspace, and I get the same underwhelming results:

image image
2023-12-07T09:40:04.095Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:04.095Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T09:40:04.095Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T09:40:10.995Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 6900 ms
2023-12-07T09:40:11.843Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:11.845Z [INFO] [streamChoices] request done: headerRequestId: [495af48b-d254-4707-b4ca-6d6ec92b2cd7] model deployment ID: [x74a4b0e7ace1]
2023-12-07T09:40:16.861Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:16.861Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T09:40:16.861Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T09:40:20.935Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 4073 ms
2023-12-07T09:40:22.073Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:22.074Z [INFO] [streamChoices] request done: headerRequestId: [80d83940-b951-4aa2-9d7e-4826919ef21c] model deployment ID: [x74a4b0e7ace1]
2023-12-07T09:40:22.183Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:22.183Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-12-07T09:40:22.184Z [INFO] [chat fetch] chat model gpt-3.5
2023-12-07T09:40:22.637Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 453 ms
2023-12-07T09:40:22.638Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:22.638Z [INFO] [streamChoices] request done: headerRequestId: [339469fc-195c-42e3-a3a9-60c72d33239a] model deployment ID: [x202bdb0a76e7]
mjbvz commented 7 months ago

@alessandro-newzoo Thanks for checking. Can you share your project or a minimal example one?

Also created a new issue for multi root support because that definitely won't work: https://github.com/microsoft/vscode-copilot-release/issues/645

piotrpalek commented 7 months ago

I have updated vscode today and all extensions, after that I've got a different error, something like "problem while collecting workspace data: failed" (sorry didn't copy). After that I've created a new chat by clicking the "+" icon on the top, and now it's working!

alessandro-newzoo commented 7 months ago

@alessandro-newzoo Thanks for checking. Can you share your project or a minimal example one?

Also created a new issue for multi root support because that definitely won't work: #645

Hi thanks for your help. The code is private but I've invited you as a maintainer, does that work for you?

On a side note, I've updated everything too like @piotrpalek did, but I didn't have his luck - still facing the same issue :(

Pager07 commented 6 months ago

fix for this issue: restart the chat. (worked for me)

alessandro-newzoo commented 6 months ago

fix for this issue: restart the chat. (worked for me)

Appreciate the input but that does not fix it for me, obviously I have restarted the chat countless times in the month and a half after opening this issue.

ZetiMente commented 6 months ago
Screenshot 2023-12-15 at 11 01 41 AM

Similar issue but here, the workspace can't determine what is a workspace is.

fatherofinvention commented 6 months ago

Doesn't work for me either. I get this reply from Copilot: "I'm sorry for any confusion, but as an AI, I don't have access to your specific workspace or codebase"

Tobeyforce commented 6 months ago

As for me I'm getting similar results as above, in addition to copilot completely hallucinating what functions certain files contain. It just seems very buggy at the moment.

mjbvz commented 6 months ago

Please test out the next insiders release of the vscode-chat extension. There was a bug that caused php identifiers to not get tokenized correctly because they start with $

kgnfth commented 6 months ago

Currently copilot isn't multi-workspace aware. I've updated the issue title to track this By default it seems like we should search the entire workspace while potentially also providing a way to scope the question to individual workspace root folders

I have removed the 2nd folder from my workspace and I still face the same issue, so I doubt it's got anything to do with my workspace being multi-root.

image image

2023-12-07T08:12:01.754Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:01.754Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T08:12:01.754Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T08:12:08.192Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 6437 ms
2023-12-07T08:12:10.286Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:10.287Z [INFO] [streamChoices] request done: headerRequestId: [55dc51a7-ac4d-453d-bfce-386a92affd4d] model deployment ID: [x74a4b0e7ace1]
2023-12-07T08:12:12.565Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:12.565Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T08:12:12.565Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T08:12:15.839Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 3274 ms
2023-12-07T08:12:17.085Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:17.086Z [INFO] [streamChoices] request done: headerRequestId: [67930983-20f3-4c5e-9960-9d9ba81ffec3] model deployment ID: [x74a4b0e7ace1]
2023-12-07T08:12:17.201Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T08:12:17.201Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-12-07T08:12:17.201Z [INFO] [chat fetch] chat model gpt-3.5
2023-12-07T08:12:17.653Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 451 ms
2023-12-07T08:12:17.654Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T08:12:17.654Z [INFO] [streamChoices] request done: headerRequestId: [60f325d9-550c-407c-83fb-9061f1459f76] model deployment ID: [x202bdb0a76e7]

I've also just tried with ALL extensions disabled except for Copilot, on a single-root local workspace, and I get the same underwhelming results:

image image

2023-12-07T09:40:04.095Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:04.095Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T09:40:04.095Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T09:40:10.995Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 6900 ms
2023-12-07T09:40:11.843Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:11.845Z [INFO] [streamChoices] request done: headerRequestId: [495af48b-d254-4707-b4ca-6d6ec92b2cd7] model deployment ID: [x74a4b0e7ace1]
2023-12-07T09:40:16.861Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:16.861Z [INFO] [chat fetch] modelMaxTokenWindow 4096
2023-12-07T09:40:16.861Z [INFO] [chat fetch] chat model gpt-4
2023-12-07T09:40:20.935Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 4073 ms
2023-12-07T09:40:22.073Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:22.074Z [INFO] [streamChoices] request done: headerRequestId: [80d83940-b951-4aa2-9d7e-4826919ef21c] model deployment ID: [x74a4b0e7ace1]
2023-12-07T09:40:22.183Z [INFO] [chat fetch] engine https://api.githubcopilot.com/chat
2023-12-07T09:40:22.183Z [INFO] [chat fetch] modelMaxTokenWindow 8192
2023-12-07T09:40:22.184Z [INFO] [chat fetch] chat model gpt-3.5
2023-12-07T09:40:22.637Z [INFO] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 453 ms
2023-12-07T09:40:22.638Z [INFO] [streamMessages] message 0 returned. finish reason: [stop]
2023-12-07T09:40:22.638Z [INFO] [streamChoices] request done: headerRequestId: [339469fc-195c-42e3-a3a9-60c72d33239a] model deployment ID: [x202bdb0a76e7]

I'm curious about the VSCode theme and font you're using, it looks really cool. Could you please share your preferences?

alessandro-newzoo commented 6 months ago

@kgnfth hey sorry for the late reply I was on holiday! The theme is https://marketplace.visualstudio.com/items?itemName=WollaceBuarque.tailwind-theme (I'm using the Darkest variation), and the font is Hasklug Nerd Font Propo: https://github.com/ryanoasis/nerd-fonts/releases/download/v3.1.1/Hasklig.zip

alessandro-newzoo commented 6 months ago

Please test out the next insiders release of the vscode-chat extension. There was a bug that caused php identifiers to not get tokenized correctly because they start with $

What version specifically? I'm using Copilot Chat v0.12.2023120701 (pre-release) and Copilot v1.147.0 (stable) on Code 1.85.1, and I still face the same issue: image

I've also tried Chat ~v0.12.2023120701~ v0.12.2024010901 on Code 1.86.0-insider , but it's been hanging on "Generating workspace info" for half an hour now

image

Can you please reopen and not close the issue until confirmed to be actually resolved? Thanks 🙏🏻

mjbvz commented 6 months ago

Please try updating. Those release are from early December of latest year. The current release is 0.12.2024010901

alessandro-newzoo commented 5 months ago

@mjbvz sorry I pasted the wrong version for Insiders, I'm indeed running Chat v0.12.2024010901 on 1.86.0-insider and it hangs indefinitely on "Gathering workspace info". I have all extensions disabled except for Copilot and Copilot Chat, and here's the output:

2024-01-10 09:52:38.323 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-01-10 09:52:38.323 [info] [chat fetch] modelMaxTokenWindow 4096
2024-01-10 09:52:38.323 [info] [chat fetch] chat model gpt-4
2024-01-10 09:52:39.440 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1117 ms
2024-01-10 09:52:43.853 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-01-10 09:52:43.854 [info] [streamChoices] request done: requestId: [cae7d24e-5afb-4042-9f01-dfbdcefad63a] responseId: [6d858429-c61a-4f02-aefd-faafe061ba1a] model deployment ID: [wda6aab3977b]
brentmjohnson commented 5 months ago

same:

2024-01-13 14:37:44.432 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions
2024-01-13 14:37:44.432 [info] [chat fetch] modelMaxTokenWindow 4096
2024-01-13 14:37:44.432 [info] [chat fetch] chat model gpt-4
2024-01-13 14:37:45.125 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 693 ms
2024-01-13 14:37:46.801 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-01-13 14:37:46.802 [info] [streamChoices] request done: requestId: [2834dcb1-7b85-4f46-ab54-48c049850796] responseId: [3deec2bb-9be2-4a23-a7f8-cb40b3abb2b9] model deployment ID: [x8d3c95500d67]
mjbvz commented 5 months ago

@alessandro-newzoo Can you share an example project which causes this issue?

alessandro-newzoo commented 5 months ago

@mjbvz the repo is private so I've added you to the contributors with Write access, but you're gonna have to accept the invite :)

It is a production repo but feel free to create a branch or fork.

thanks!

kevinLourencoDev commented 5 months ago

Hi, maybe the not yet released? I'm using the Pre-release v0.12.2024012301 and when I ask the chat: " what i can do with the @workspace command ? " the answer:

The @workspace command is a placeholder used in this context to represent actions or queries related to your current workspace in Visual Studio Code. However, it's important to note that @workspace is not a real command you can use in Visual Studio Code or any other IDE.

As an AI, I can assist you with various tasks related to your workspace, such as:

Answering general programming questions Explaining how the selected code works Generating unit tests for the selected code Proposing a fix for the problems in the selected code Scaffolding code for a new workspace Creating a new Jupyter Notebook Answering questions about VS Code Generating query parameters for workspace search Answering questions about VS Code extension development Guiding on how to do something in the terminal However, I don't have the ability to directly interact with or manipulate your workspace.

It's not look like it will be capable of doing anything with stuff that are not opened files...

AA-ATL commented 5 months ago

Something that worked for me was I had to first open a folder via "File" > "Open Folder" instead of just navigating to a given folder via the terminal cli. This allowed "@workspace" context.