search

microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
326 stars 30 forks source link

Response Filtered #1116

Open craftzneko opened 6 months ago

craftzneko commented 6 months ago

image

image

image

Steps to Reproduce:

  1. See images

I am getting many more errors in copilot over the last 6 months, this is an example. I would estimate 1 in 10 requests i make either fail outright or return incorrect information. Perhaps i am seeking to much but it feels objectively worse as a co pilot from a year ago. Searching for the exact same result works

craftzneko commented 6 months ago

Also just to add in terms of UI. When a response gets filtered ignoring if it is correct or not the grey filter affect is really annoying and personally just adds to my frustration using it. It almost feels like its teasing you with a result that you could use 🤣

lramos15 commented 6 months ago

If you an include your logs from CTRL/ CMD + SHIFT + U -> GitHub Copilot Chat that can help us tell you what filter we're hititing.

The greying out is a requirement of the filtration, we have actively been investigating these but our data generally does not show a high filtration rate, your use case may just be susceptible to filters

craftzneko commented 5 months ago

Happened again this time on the prerelease. Here is my text and prompt and chat log image

2024-04-26 10:53:12.769 [info] [streamMessages] message 0 returned. finish reason: [stop] 2024-04-26 10:53:12.769 [info] [streamChoices] request done: requestId: [a65c6e46-63a7-4972-9be9-32d3e2b7b535] model deployment ID: [x4dff5e5d11fc] 2024-04-26 11:14:25.692 [info] [auth] Logged in as craftzneko 2024-04-26 11:14:26.947 [info] [chat] copilot token chat_enabled: true 2024-04-26 11:14:26.947 [info] [auth] Got Copilot token for craftzneko 2024-04-26 11:39:26.916 [info] [auth] Logged in as craftzneko 2024-04-26 11:39:27.792 [info] [chat] copilot token chat_enabled: true 2024-04-26 11:39:27.792 [info] [auth] Got Copilot token for craftzneko 2024-04-26 11:46:36.613 [info] [chat fetch] url https://api.githubcopilot.com/chat/completions 2024-04-26 11:46:36.613 [info] [chat fetch] modelMaxPromptTokens 7168 2024-04-26 11:46:36.613 [info] [chat fetch] modelMaxResponseTokens 2142 2024-04-26 11:46:36.613 [info] [chat fetch] chat model gpt-4-0125-adaf089e 2024-04-26 11:46:42.813 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 6200 ms 2024-04-26 11:46:42.902 [info] [streamMessages] message 0 returned. finish reason: [content_filter] 2024-04-26 11:46:42.903 [info] [streamChoices] request done: requestId: [c1d5d7d2-f5b6-4e04-93ca-15c8fde516c0] model deployment ID: []

taurisjoe commented 5 months ago

I was chatting with Copilot about calculating cumulative values Chat was not too happy with the shortened version of that.