.Net: Bug: Exception occurs at the end of foreach when executing GetStreamingChatMessageContentsAsync

[hanhsia]

I encounter the same issue as the bug #8286 . But it's closed. So i created a new one. the code works fine until I enabled Asynchronous Filter in content filter of gpt-4o model. This is call stack:

This is the http response from gpt-4o: data: {"choices":[],"created":0,"id":"","model":"","object":"","prompt_filter_results":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"jailbreak":{"filtered":false,"detected":false},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}]}

data: {"choices":[{"delta":{"content":"","role":"assistant"},"finish_reason":null,"index":0,"logprobs":null}],"created":1724860848,"id":"chatcmpl-A1FOCU4vamBSxxqjaKcvADYMlwlaC","model":"gpt-4o-2024-05-13","object":"chat.completion.chunk","system_fingerprint":"fp_abc28019ad"}

data: {"choices":[{"delta":{"content":"善"},"finish_reason":null,"index":0,"logprobs":null}],"created":1724860848,"id":"chatcmpl-A1FOCU4vamBSxxqjaKcvADYMlwlaC","model":"gpt-4o-2024-05-13","object":"chat.completion.chunk","system_fingerprint":"fp_abc28019ad"} ... data: {"choices":[{"delta":{},"finish_reason":"stop","index":0,"logprobs":null}],"created":1724860848,"id":"chatcmpl-A1FOCU4vamBSxxqjaKcvADYMlwlaC","model":"gpt-4o-2024-05-13","object":"chat.completion.chunk","system_fingerprint":"fp_abc28019ad"}

data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2318},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2318},"content_filter_results":{"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

data: [DONE]

[hanhsia]

compared the raw data between aync filter mode and standard mode. The difference is there are additional data received which doesn't include any delta payload in async filter mode: data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2297},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

data: {"choices":[{"content_filter_offsets":{"check_offset":1576,"start_offset":1576,"end_offset":2297},"content_filter_results":{"protected_material_code":{"filtered":false,"detected":false},"protected_material_text":{"filtered":false,"detected":false}},"finish_reason":null,"index":0}],"created":0,"id":"","model":"","object":""}

[RogerBarreto]

@hanhsia Thanks for all the valuable additional context, investigating.

[RogerBarreto]

@hanhsia I was able to reproduce the error and seems to be a bug happening in the OpenAI SDK for those chunks, depending on their response for this, we may wait or apply a temporary fix on our side.

Opened the Issue in their repo for investigation

• https://github.com/openai/openai-dotnet/issues/198

[JP-droidic]

@RogerBarreto, we're experiencing the same problem, and it happens randomly, depending on what OpenAi returns.

Can you please apply a temporary fix ASAP as otherwise 1.18.x is not stabile and ready to be released

thanks guys

[RogerBarreto]

Fixed in the #8422