ai-genie / chatgpt-vscode

Your best AI pair programmer in VS Code
https://marketplace.visualstudio.com/items?itemName=genieai.chatgpt-vscode
ISC License
1.21k stars 74 forks source link

Editor mode doesn't use GPT4 (defaults to text-davinci-003) #24

Open pistachiomatt opened 1 year ago

pistachiomatt commented 1 year ago

Verify it's not a duplicate bug report

Describe the Bug

  1. Change genieai.openai.model in settings.json to GPT4
  2. Confirm GPT4 is working in conversation view by observing speed and eloquence response
  3. Switch to editor mode and prompt a question. The response speed is way too fast to be GPT4; it must still be using gpt-3.5-turbo

Please tell us if you have customized any of the extension settings or whether you are using the defaults.

The genieai.openai.model is already set to GPT4, and the conversation view does successfully use GPT4.

Additional context

No response

genieai-info commented 1 year ago

Hello @pistachiomatt - the editor view is currently defaulted to text-davinci-003 for various reasons:

  1. text-davinci-003 is the only model that'd officially respect the stop parameter to ensure you are getting a code response.
  2. The nature of Editor View is not conversational but iterating within a context. So the initial implementation was done using davinci model

Though, we are still considering switching the Editor View to respect your model choice and provide a better messaging possible for the best experience. That is, if we could find a way to ensure GPT-3.5 / GPT-4 can really respond only using code (Which is relevant since otherwise you will get bunch of linter errors in Editor with non-code/non-commented text)

pistachiomatt commented 1 year ago

You can get GPT4 to follow a format. You can prompt it to "only respond with a code block" or "respond with this JSON format: { full_code: "<code only answer that…>" }". Lowering the temperature might also help.

cmp-nct commented 1 year ago

davinci-003 is a magnitude more expensive than 3.5-turbo and less capable in my tests. all you need is a proper prompt and a stop condition inside your code instead of using stop words. Just drop the connection once stop is reached.

genieai-info commented 1 year ago

@cmp-nct @pistachiomatt thank you! we are aware of it and working on the best solution to provide customizability of models in Editor View. Please bear with us and we'll keep here posted once we rollout an update 🙏🏽

veonua commented 1 year ago

Azure OpenAI Service: Unsupported data type Error: This means you didn't select the right base model for your Azure OpenAI service. Make sure to select the base GPT model you used to create your Azure OpenAI Service deployment

well, if the editor is not using GPT4. What do we do in Azure then, should we setup two endpoints to have conv and editor mode running?

GPT4 is way superior in code-generation tasks

LL1yt commented 1 year ago

Is it possible to add a setting that will switch the model for editor view (let there be a warning for potential problems)? I'm willing to accept temporary inconvenience, but on the other hand I can use gpt-4 at my own risk in this mode.

alanwilter commented 1 year ago

I'd definitely love to have GPT4 in the Editor view. And to what extension can GPT4 API be compared to Chat GPT Plus? Is there a way of passing images?

cloudbring commented 10 months ago

This is still broken for commit message even if you change the model to GPT4 (mine is currently gpt-4-1106-preview) but, commit message still uses text-davinici-003

martinguenther commented 10 months ago

text-davinici-003 is deprecated since 2024-01-04 (see https://platform.openai.com/docs/deprecations/instructgpt-models). I get the following error when working in editor mode: An error occured. ChatGPT error 404: { "error": { "message": "The modeltext-davinci-003has been deprecated, learn more here: https://platform.openai.com/docs/deprecations", "type": "invalid_request_error", "param": null, "code": "model_not_found" } } This seems related to #155. Did I miss something, or is the editor mode currently not usable due to it being defaulted text-davinci-003?

cmp-nct commented 10 months ago

Just use continue.dev, they are far better than Genie

murali-marimekala commented 8 months ago

text-davinici-003 is deprecated since 2024-01-04 (see https://platform.openai.com/docs/deprecations/instructgpt-models). I get the following error when working in editor mode: An error occured. ChatGPT error 404: { "error": { "message": "The modeltext-davinci-003has been deprecated, learn more here: https://platform.openai.com/docs/deprecations", "type": "invalid_request_error", "param": null, "code": "model_not_found" } } This seems related to #155. Did I miss something, or is the editor mode currently not usable due to it being defaulted text-davinci-003?

I am observing the same issue. Any help ?

5tigerjelly commented 5 months ago

same issue, I guess it isn't maintained anymore?