Copilot makes invalid suggests seemingly using prior chat history instead of the current chat

sean-mcmanus commented 2 weeks ago

Type: Bug

Copilot says "Changed "Fix an issing" but no such text exists in my submitted text. It appears to incorrectly be using my prior chat text (which did have that error, but I fixed it).

See the screenshot:

The "reference" was an unrelated file that I don't think is related to the bug.

I asked it to answer the question without using the prior chat history but it gave the exact same response.

Extension version: 0.21.0 VS Code version: Code 1.94.0 (d78a74bcdfad14d5d3b1b782f87255d802b57511, 2024-10-02T13:08:12.626Z) OS version: Windows_NT x64 10.0.26100 Modes:

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz (8 x 2112)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off| |Load (avg)|undefined| |Memory (System)|15.92GB (5.55GB free)| |Process Argv|--crash-reporter-id 45c5b3ab-1ded-44bd-80c7-d6d1aff931f6| |Screen Reader|no| |VM|0%|

A/B Experiments

``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscod805:30301674 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 vscaac:30438847 c4g48928:30535728 azure-dev_surveyone:30548225 vscrpc:30673769 a9j8j154:30646983 962ge761:30959799 pythongtdpath:30769146 welcomedialogc:30910334 pythonnoceb:30805159 asynctok:30898717 pythonmypyd1:30879173 h48ei257:31000450 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 dsvsc021:30996838 945dj816:31013170 a69g1124:31058053 dvdeprecation:31068756 dwnewjupyter:31046869 newcmakeconfigv2:31071590 impr_priority:31102340 nativerepl2:31139839 refactort:31108082 pythonrstrctxt:31112756 flightc:31134773 wkspc-onlycs-t:31132770 wkspc-ranged-t:31151552 cf971741:31144450 autoexpandse:31146404 iacca2:31150323 notype1cf:31151524 cc771715:31146322 ```

sean-mcmanus commented 2 weeks ago

UPDATE: I got Copilot to admit it made a mistake via a series of follow up questions:

sean-mcmanus commented 3 days ago

I've reproed this bug again -- i.e. it appears to be a reproducable bug.

It seems to incorrectly be suggesting stale suggestions and I'm unable to get it to ignore the previous text I provided and to use the new text I'm providing. The text in the CHANGELOG.md context also matches what I'm currently trying to get it to process.

microsoft / vscode-copilot-release

Copilot makes invalid suggests seemingly using prior chat history instead of the current chat #1762