tabAutocompleteModel raise http 400 with litellm

[goodosoft]

Before submitting your bug report

• [X] I believe this is a bug. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that reports the same bug
• [X] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: macOS 14.5
- Continue: v0.8.40
- IDE: Visual Studio Code 1.90.2

{"error":{"message":"litellm.BadRequestError: BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': \"Invalid 'stop': array too long. Expected an array with maximum length 4, but got an array with length 14 instead.\", 'type': 'invalid_request_error', 'param': 'stop', 'code': 'array_above_max_length'}}","type":null,"param":null,"code":400}}

Description

{"error":{"message":"litellm.BadRequestError: BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': \"Invalid 'stop': array too long. Expected an array with maximum length 4, but got an array with length 14 instead.\", 'type': 'invalid_request_error', 'param': 'stop', 'code': 'array_above_max_length'}}","type":null,"param":null,"code":400}}

To reproduce

No response

Log output

{"error":{"message":"litellm.BadRequestError: BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': \"Invalid 'stop': array too long. Expected an array with maximum length 4, but got an array with length 14 instead.\", 'type': 'invalid_request_error', 'param': 'stop', 'code': 'array_above_max_length'}}","type":null,"param":null,"code":400}}

[sestinj]

@goodosoft This is an edge case we haven't considered largely because GPT-4 isn't intended for autocomplete. Do you have access to any other models, like Codestral? It will be cheaper, faster, more accurate, give correct formatting, and won't cause this problem

[goodosoft]

@goodosoft This is an edge case we haven't considered largely because GPT-4 isn't intended for autocomplete. Do you have access to any other models, like Codestral? It will be cheaper, faster, more accurate, give correct formatting, and won't cause this problem

Thanks, we will try Codestral with ollama.

[goodosoft]

use Cloudflare Workers AI as tabAutocompleteModel, it works. But sometimes show <｜end▁of▁sentence｜> .

1630