Error generating autocompletion with Qwen2.5-Coder-7B and vllm

[LNTH]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Windows 11
- Continue: v0.9.211 (pre-release)
- IDE: VS-Code
- Model: `Qwen/Qwen2.5-Coder-7B` served with `vllm`
- config.json:

  {
    "models": [],
    "tabAutocompleteModel": {
      "title": "Qwen/Qwen2.5-Coder-7B",
      "provider": "vllm",
      "model": "Qwen/Qwen2.5-Coder-7B",
      "apiBase": "http://192.168.1.19:8000/v1",
      "apiKey": "None",
      "completionOptions": {
        "template": "<|fim_prefix|>{{{ prefix }}}<|fim_suffix|>{{{ suffix }}}<|fim_middle|>",
        "stop": ["<|endoftext|>"]
      }
    }
  }

Description

The tabAutoComplete feature is not displaying any suggestions in the VS Code editor.

• Continue TabAutoComplete is enabled
• VSCode: Inline Suggestion is enabled

To reproduce

1. Ensure the vllm server is running. Confirmed by observing the log entry: "GET /v1/models HTTP/1.1" 200 OK whenever the config.json is modified.
2. Type in the VS Code editor to trigger auto-completion.

Expected Behavior

Auto-completion suggestions should appear in the VS Code editor.

Actual Behavior

vllm server received "POST /v1/completions HTTP/1.1" 200 OK but nothing show on VsCode Editor. VsCode Console displayed Error generating autocompletion: TypeError: Cannot read properties of undefined (reading 'includes')

Additional Observations

After this error occurs, the Continue extension no longer sends POST /v1/completions requests to the vllm server.

Log output

[Extension Host] Error generating autocompletion: TypeError: Cannot read properties of undefined (reading 'includes')
    at c:\Users\MyUser\.vscode\extensions\continue.continue-0.9.211-win32-x64\out\extension.js:102778:87
    at Array.some (<anonymous>)
    at _CompletionProvider.getTabCompletion (c:\Users\MyUser\.vscode\extensions\continue.continue-0.9.211-win32-x64\out\extension.js:102778:61)
    at async _CompletionProvider.provideInlineCompletionItems (c:\Users\MyUser\.vscode\extensions\continue.continue-0.9.211-win32-x64\out\extension.js:102697:27)
    at async ContinueCompletionProvider.provideInlineCompletionItems (c:\Users\MyUser\.vscode\extensions\continue.continue-0.9.211-win32-x64\out\extension.js:517910:27)
    at async Y.provideInlineCompletions (c:\Users\MyUser\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:161:123619)

[7216]

Ran into this same issue.

[7216]

Found a workaround to get some completions.

{
  "models": [
    {
      "title": "Qwen2.5-Coder-7b-Instruct",
      "provider": "vllm",
      "model": "Orion-zhen/Qwen2.5-Coder-7B-Instruct-AWQ",
      "apiBase": "http://10.0.0.10:8000/v1"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder-7b-Instruct",
    "provider": "openai",
    "apiKey": "None",
    "completionOptions": {
      "stop": [
        "<|endoftext|>",
        "\n"
      ]
    },
    "apiBase": "http://10.0.0.10:8000/v1/",
    "model": "Orion-zhen/Qwen2.5-Coder-7B-Instruct-AWQ"
  },
  "tabAutocompleteOptions": {
    "multilineCompletions": "never",
    "template": "You are a helpful assistant.<|fim_prefix|>{{{ prefix }}}<|fim_suffix|>{{{ suffix }}}<|fim_middle|>"
  },
  "customCommands": [],
  "allowAnonymousTelemetry": false,
  "docs": []
}

Namely the tabAutocompleteOptions template, and the model provider being openai with the completion options stop including the two entries.