FIM return escaping symbols

Semihal commented 4 days ago

Before submitting your bug report

[X] I believe this is a bug. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that reports the same bug
[X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: MacOS 14.5
- Continue: v0.8.42
- IDE: VSCode 1.90.2
- Model: vLLM (CodeQwen1.5-7B) -> LiteLLM
- config.json:

{
  "tabAutocompleteModel": {
    "title": "Autocomplete model",
    "provider": "openai",
    "apiBase": "<endpoint>",
    "apiKey": "<token>",
    "model": "qwen",
    "useLegacyCompletionsEndpoint": true,
    "requestOptions": {
      "verifySsl": false
    }
  },
  "tabAutocompleteOptions":{
    "useSuffix": false,
    "useCache": false,
    "template": "<fim_prefix>{{ prefix }}<fim_suffix>{{ suffix }}<fim_middle> "
  },
  "allowAnonymousTelemetry": false
}

Description

I had broken autocomplete:

The same model with the CodeGPT plugin for JetBrainns is working properly :(

To reproduce

No response

Log output

No response

sestinj commented 4 days ago

@Semihal can you share what you see in the prompt logs? This would help us know whether the model is sending these escaped characters or not

Semihal commented 3 days ago

@Semihal can you share what you see in the prompt logs? This would help us know whether the model is sending these escaped characters or not

==========================================================================
Settings:
contextLength: 4096
model: qwen-fim
maxTokens: 1024
temperature: 0.01
raw: true
stop: 

,

,/src/,#- coding: utf-8,```,
def,
class,
"""#
log: undefined

############################################

<fim_prefix>    for x in snake_list:
        pygame.draw.rect(dis, black, [x[0], x[1], snake_block, snake_block])

def gameLoop():
    game_over &#x3D; False
    game_close &#x3D; False

    x1 &#x3D; dis_width / 2
    y1 &#x3D; dis_height / 2

    x1_change &#x3D; 0
    y1_change &#x3D; 0

    snake_list &#x3D; []
    Length_of_snake &#x3D; 1

    foodx &#x3D; round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
    foody &#x3D; round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0

    while not game_over:

        while game_close &#x3D;&#x3D; True:
            dis.fill(blue)
            message(&quot;Игра окончена. Q-Выход, C-Продолжить&quot;, red)
            pygame.display.update()

            for event in pygame.event.get():
                if event.type &#x3D;&#x3D; pygame.KEYDOWN:
                    if event.key &#x3D;&#x3D; pygame.K_q:
                        game_over &#x3D; True
                        game_close &#x3D; False
                    if event.key &#x3D;&#x3D; pygame.K_c:
                        gameLoop()

        for event in pygame.event.get():
            if event.type &#x3D;&#x3D; pygame.QUIT:
                game_over &#x3D; True
                game_close &#x3D; False
            if event.type &#x3D;&#x3D; pygame.KEYDOWN:
                if event.key &#x3D;&#x3D; pygame.K_UP:
                    y1_change &#x3D; -snake_block
                    x1_change &#x3D; 0
                elif event.key &#x3D;&#x3D; pygame.K_DOWN:
                    y1_change &#x3D; snake_block
                    x1_change &#x3D; 0
                elif event.key &#x3D;&#x3D; pygame.K_LEFT:
                    x1_change &#x3D; -snake_block
                    y1_change &#x3D; 0
                elif event.key &#x3D;&#x3D; pygame.K_RIGHT:
                    x1_change &#x3D; snake_block
                    y1_change &#x3D; 0

        if x1 &gt;&#x3D; dis_width or x1 &lt; 0 or y1 &gt;&#x3D; dis_height or y1 &lt; 0:
            game_close &#x3D; True
        x1 +&#x3D; x1_change
        y1 +&#x3D; y1_change
        dis.fill(blue)
        pygame.draw.rect(dis, green, [foodx, foody, snake_block, snake_block])
        snake_Head &#x3D; []
        snake_Head.append(x1)
        snake_Head.append(y1)
        snake_list.append(snake_Head)
        if len(snake_list) &gt; Length_of_snake:
            del snake_list[0]

        for x in snake_list[:-1]:
            if x &#x3D;&#x3D; snake_Head:
                game_close &#x3D; True

        our_snake(snake_block, snake_list)

        pygame.display.update()

        if x1 &#x3D;&#x3D; foodx and y1 &#x3D;&#x3D; foody:
            foodx &#x3D; round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
            foody &#x3D; round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0
            Length_of_snake +&#x3D; 1

        if <fim_suffix>

        clock.tick(snake_speed)

    pygame.quit()
    quit()<fim_middle> ==========================================================================
==========================================================================
Completion:

0 &lt;= x1 &lt; dis_width and 0 &lt;= y1 &lt; dis_height:
            pygame.draw.rect(dis, black, [x1, y1, snake_block, snake_block])
        else:
            game_close &#x3D; True

As I understand it, there is excessive shielding, to which LM responds in the same way?

sestinj commented 3 days ago

@Semihal is it safe to assume that your code doesn't actually look like &#x3D and that these should all be =? It looks like the language model is just responding to match the escape pattern that it sees above, so the problem is actually in the prompt

Semihal commented 3 days ago

@Semihal is it safe to assume that your code doesn't actually look like &#x3D and that these should all be =? It looks like the language model is just responding to match the escape pattern that it sees above, so the problem is actually in the prompt

I have the code without any escaping. The screenshot in the original message shows an example (there is no escaping). Apparently, the plugin is running a data transfer escape and LLM is trying to match it. Can we make some additional parameter in config.json to disable the escaping?

continuedev / continue