openai / openai-dotnet

The official .NET library for the OpenAI API
https://www.nuget.org/packages/OpenAI
MIT License
1.53k stars 156 forks source link

Stream mode function parameters parse incorrect. #256

Closed jichulu closed 1 month ago

jichulu commented 1 month ago

Service

OpenAI

Describe the bug

https://github.com/openai/openai-dotnet/blob/ef201cd6016bc1111be4c0098fd643554d6e9a0b/src/Generated/Models/InternalChatCompletionMessageToolCallChunkFunction.Serialization.cs#L99

This code to parse sse data will add null parameters (string name = null; BinaryData arguments = null;) to InternalChatCompletionMessageToolCallChunkFunction When to call StreamingChatToolCallUpdate.FunctionArgumentsUpdate.ToString() will throw null exception.

It may be cause by this SSE data. The delta is empty

data: {"id":"xx",...,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

Steps to reproduce

See error

Log

Trace: [2024/10/11 21:25:51] Request: POST https://xxx.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview
Accept: application/json
User-Agent: azsdk-net-AI.OpenAI/2.1.0-beta.1, (.NET 8.0.8; Microsoft Windows 10.0.22631)
x-ms-client-request-id: 0ae8cafc-d86d-48bb-a39c-b0ce1c7f9bc0
api-key: ***
Content-Type: application/json

{"messages":[{"role":"system","content":"You are a smart weather assistant"},{"role":"user","content":"Guangzhou weather"}],"model":"gpt-4o","frequency_penalty":0,"presence_penalty":0,"stream":true,"temperature":0.7,"top_p":1,"tools":[{"type":"function","function":{"description":"Get city weather based on area code","name":"WeatherPlugin-GetWeather","parameters":{"type":"object","required":["addressCode"],"properties":{"addressCode":{"type":"string","description":"City area address code. e.g. 110000, 440100..."}}}}}],"tool_choice":"auto","max_tokens":2048}

Trace: [2024/10/11 21:25:53] /openai/deployments/gpt-4o/chat/completions OK
Transfer-Encoding: chunked
...
Content-Type: text/event-stream; charset=utf-8

Trace: [2024/10/11 21:25:53] 
data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"role":"assistant","content":null,"tool_calls":[{"index":0,"id":"call_yXkmOQ1GvYheU9tVolJngmLa","type":"function","function":{"name":"WeatherPlugin-GetWeather","arguments":""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\n"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":" "}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"address"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"Code"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":" \""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"440"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"100"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"\n"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"xxx","object":"chat.completion.chunk","created":1728653152,"model":"gpt-4o-2024-05-13","system_fingerprint":"fp_67802d9a6d","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}]}

data: [DONE]

Error: [2024/10/11 21:25:53]
System.ArgumentNullException: Value cannot be null. (Parameter 'bytes')
   at System.ArgumentNullException.Throw(String paramName)
   at System.Text.Encoding.GetString(Byte* bytes, Int32 byteCount)
   at System.BinaryData.ToString()
   at Microsoft.SemanticKernel.Connectors.OpenAI.OpenAIFunctionToolCall.TrackStreamingToolingUpdate(IReadOnlyList`1 updates, Dictionary`2& toolCallIdsByIndex, Dictionary`2& functionNamesByIndex, Dictionary`2& functionArgumentBuildersByIndex)
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(String targetModel, ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+MoveNext()
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(String targetModel, ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+MoveNext()
   at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(String targetModel, ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
...

Code snippets

No response

OS

winos

.NET version

.net 8.0

Library version

2.1.0-beta.1

colincmac commented 1 week ago

For those that find this error, if you explicitly add System.Memory.Data version 9 to your project, this error seems resolved.