CoderCookE / vim-chatgpt

Vim Plugin For ChatGPT
The Unlicense
230 stars 29 forks source link

Handle empty choices in streamed chunks #37

Closed armanschwarz closed 5 months ago

armanschwarz commented 5 months ago

Newer Azure API responses contain an empty first chunk when streaming. For example, take the below code:

client = AzureOpenAI(
    azure_endpoint=azure_endpoint,
    azure_deployment=azure_deployment,
    api_key=api_key,
    api_version='2024-02-15-preview'
)

response = client.chat.completions.create(
    model='gpt-3.5-turbo',
    messages=[
        {
            'role': 'system',
            'content': 'You are a helpful expert programmer we are working together to solve complex coding challenges, and I need your help. Please make sure to wrap all code blocks in ``` annotate the programming language you are using. '
        },
        {
            'role': 'user',
            'content': 'Hello'
        }],
    temperature=0.0,
    max_tokens=4096,
    stop='',
    stream=True
)

for chunk in response:
    print(chunk)
    break

This outputs:

ChatCompletionChunk(id='', choices=[], created=0, model='', object='', system_fingerprint=None, prompt_filter_results=[{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'jailbreak': {'filtered': False, 'detected': False}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}])

This results in exceptions in the current vim-chatgpt when the plugin tries to index the 0th element of the choices array.

Skipping empty chunk responses solves this.

CoderCookE commented 5 months ago

Looks good, i don't have access to the Azure Version so thanks for catching this!