Newer Azure API responses contain an empty first chunk when streaming. For example, take the below code:
client = AzureOpenAI(
azure_endpoint=azure_endpoint,
azure_deployment=azure_deployment,
api_key=api_key,
api_version='2024-02-15-preview'
)
response = client.chat.completions.create(
model='gpt-3.5-turbo',
messages=[
{
'role': 'system',
'content': 'You are a helpful expert programmer we are working together to solve complex coding challenges, and I need your help. Please make sure to wrap all code blocks in ``` annotate the programming language you are using. '
},
{
'role': 'user',
'content': 'Hello'
}],
temperature=0.0,
max_tokens=4096,
stop='',
stream=True
)
for chunk in response:
print(chunk)
break
Newer Azure API responses contain an empty first chunk when streaming. For example, take the below code:
This outputs:
This results in exceptions in the current
vim-chatgpt
when the plugin tries to index the 0th element of thechoices
array.Skipping empty chunk responses solves this.