search

microsoft / vscode-copilot-release

Feedback on GitHub Copilot Chat UX in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat
Creative Commons Attribution 4.0 International
340 stars 36 forks source link

Copilot not really being an ai 😆 #605

Open outgaze opened 11 months ago

outgaze commented 11 months ago

Steps to Reproduce:

  1. Go to leet code
  2. Find a hard question with low acceptance rate
  3. When github copilot or chat provides wrong solution provide it with the test cases it's solution is failing
  4. It'll start outputting same solutions and suggestions again and again failing to understand even ignoring new input 🙂
fisforfaheem commented 11 months ago

Agree samw with flutter documentation

lramos15 commented 11 months ago

This may just be a model limitation.

Can you please share the question you tried?

outgaze commented 11 months ago

This may just be a model limitation.

Can you please share the question you tried?

https://leetcode.com/problems/booking-concert-tickets-in-groups/ it's not only that one though it's most of the low acceptance rate questions medium to hard difficulty if it's a model limitation then that's pretty bad cause if we're trying to create something out of a stereotype it means that copilot won't be helpful at all

robertgro commented 11 months ago

Experienced this with other services as well. Are the chat bots that learn from user input only for those who can afford it? The rest gets wrong output without the bot being able to learn? If so, then chatbots are useless and a waste of time for most of the users.

Edit: After playing around with Copilot a bit, it starts to become helpful again. Let's see!