"<COMPLETION>" in autocomplete output

canadiannomad commented 1 month ago

Before submitting your bug report

[X] I believe this is a bug. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that reports the same bug
[X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: macOS 12.7.4
- Continue: v0.9.132
- IDE:1.90.0-insider

  "tabAutocompleteModel": {
    "title": "GPT-4o",
    "provider": "openai",
    "model": "gpt-4o",
    "apiKey": "...",
  }

Description

Every time I try to get a completion the output starts with "...the code that could be inserted"

Here is what I see in the logs:

TASK: Fill the {{FILL_HERE}} hole. Answer only with the CORRECT completion, and NOTHING ELSE. Do it now.
<COMPLETION>==========================================================================
==========================================================================
Completion:

<COMPLETION>info!("Example");

And in the tab completion, I actually see the "" there.

Presumably that should be stripped.

To reproduce

Log output

This is an example of tab completion using gpt-4o <COMPLETION>model, which is a variant of the GPT-4 model developed by OpenAI.<COMPLETION> It is designed to provide more efficient and context-aware completions. <COMPLETION>It can be used for various applications, including code completion, text generation, and more. ... {{FILL_HERE}}
</QUERY>
TASK: Fill the {{FILL_HERE}} hole. Answer only with the CORRECT completion, and NOTHING ELSE. Do it now.
<COMPLETION>==========================================================================
==========================================================================
Completion:

<COMPLETION>Overall, it aims to enhance productivity and streamline workflows for developers and content creators.

kuamatzin commented 1 month ago

I have the same issue

byrondelgado commented 1 month ago

Same here

ilyanoskov commented 1 month ago

I am having the same issue

mtrevin93 commented 4 weeks ago

I am seeing this too. My autocomplete Ex:

const PRIMESTOTWOHUNDRED = {{FILL_HERE}}
</QUERY>
TASK: Fill the {{FILL_HERE}} hole. Answer only with the CORRECT completion, and NOTHING ELSE. Do it now.
<COMPLETION>==========================================================================
==========================================================================
Completion:

<COMPLETION>[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199]

We are using an anthropic model bedrock/anthropic.claude-3-sonnet-20240229-v1:0 Our API Proxy has to strip out /n and /r from the response, but then the completion tag is not trimmed out because it is not on a new line.

sestinj commented 3 weeks ago

@mtrevin93 @ilyanoskov @byrondelgado @kuamatzin @canadiannomad Now that there is an open model capable of extremely high autocomplete quality (Codestral) I would recommend using this. It is free until August, and even after will be a LOT cheaper than gpt-4. We have a set up guide here: https://docs.continue.dev/walkthroughs/set-up-codestral

I will still attempt to solve this problem, but there are likely to be many more imperfections due to the fact that gpt-4o is not at all trained for autocomplete.

mtrevin93 commented 3 weeks ago

@sestinj Thank you for your response. Unfortunately, a public API will not work for my use case - I must use an API hosted on my company's infrastructure which has access to all models supported on AWS bedrock.

continuedev / continue