microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
348 stars 37 forks source link

Completely unrelated output #3030

Open valerierose opened 7 hours ago

valerierose commented 7 hours ago

Type: Bug

Copilot is producing extra bad results today. Several times in a row it's producing text in a code cell or doing something that appears completely unrelated to the prompt.

Extension version: 0.22.4 VS Code version: Code 1.95.3 (Universal) (f1a4fb101478ce6ec82fe9627c43efbf9e98c813, 2024-11-13T14:50:04.152Z) OS version: Darwin arm64 23.6.0 Modes: Connection to 'dev-container+7b22686f737450617468223a222f55736572732f76616c2f636f64652f6e696e6465786572222c226c6f63616c446f636b6572223a66616c73652c2273657474696e6773223a7b22636f6e74657874223a226465736b746f702d6c696e7578227d2c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d' could not be established Connection to 'dev-container+7b22686f737450617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d617069222c226c6f63616c446f636b6572223a66616c73652c2273657474696e6773223a7b22636f6e74657874223a226465736b746f702d6c696e7578227d2c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d' could not be established

System Info |Item|Value| |---|---| |CPUs|Apple M3 Pro (11 x 2400)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off| |Load (avg)|4, 3, 4| |Memory (System)|36.00GB (0.06GB free)| |Process Argv|--crash-reporter-id 27c2c5c2-f09b-4edf-86f9-dd6f03898ca4| |Screen Reader|no| |VM|0%| Connection to 'dev-container+7b22686f737450617468223a222f55736572732f76616c2f636f64652f6e696e6465786572222c226c6f63616c446f636b6572223a66616c73652c2273657474696e6773223a7b22636f6e74657874223a226465736b746f702d6c696e7578227d2c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f55736572732f76616c2f636f64652f6e696e64657865722f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d' could not be established Connection to 'dev-container+7b22686f737450617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d617069222c226c6f63616c446f636b6572223a66616c73652c2273657474696e6773223a7b22636f6e74657874223a226465736b746f702d6c696e7578227d2c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f55736572732f76616c2f636f64652f6d6f6e6f7265706f2f7061636b616765732f736572766963652e6e65796e61722d6170692f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d' could not be established
A/B Experiments ``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscod805:30301674 binariesv615:30325510 vsaa593:30376534 py29gd2263:31024239 vscaac:30438847 c4g48928:30535728 azure-dev_surveyone:30548225 2i9eh265:30646982 962ge761:30959799 pythonnoceb:30805159 asynctok:30898717 pythonmypyd1:30879173 2e7ec940:31000449 pythontbext0:30879054 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 dsvsc021:30996838 f3je6385:31013174 dvdeprecation:31068756 dwnewjupyter:31046869 2f103344:31071589 impr_priority:31102340 nativerepl1:31139838 pythonrstrctxt:31112756 cf971741:31144450 iacca1:31171482 notype1cf:31157160 5fd0e150:31155592 dwcopilot:31170013 stablechunks:31181875 ```
valerierose commented 7 hours ago

My prompt was: "color code the rows of cases for printing. if the predicted score is within 0.1 of the expected score, make it green. otherwise make it red"

And the output was text instead of code: "It looks like you have a dataset X_train with various features...."