Community Issue: How to customize response handlers？

[sophialagerkranspandey]

Discussed in https://github.com/microsoft/semantic-kernel/discussions/6198

^{Originally posted by **niltor** May 12, 2024} ## Problems I encountered When I use `Kernel.InvokePromptStreamingAsync(prompt);`, I have errors: ```log System.ArgumentNullException: Value cannot be null. (Parameter 'value') at Azure.AI.OpenAI.ChatRole.op_Implicit(String value) at Azure.AI.OpenAI.StreamingChatCompletionsUpdate.DeserializeStreamingChatCompletionsUpdates(JsonElement element) at Azure.Core.Sse.SseAsyncEnumerator`1.EnumerateFromSseStream(Stream stream, Func`2 multiElementDeserializer, CancellationToken cancellationToken)+MoveNext() at Azure.Core.Sse.SseAsyncEnumerator`1.EnumerateFromSseStream(Stream stream, Func`2 multiElementDeserializer, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource.GetResult() at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(ChatHistory chat, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(ChatHistory chat, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(ChatHistory chat, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource.GetResult() at Microsoft.SemanticKernel.KernelFunctionFromPrompt.InvokeStreamingCoreAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.KernelFunctionFromPrompt.InvokeStreamingCoreAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.KernelFunctionFromPrompt.InvokeStreamingCoreAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource.GetResult() at Microsoft.SemanticKernel.KernelFunction.InvokeStreamingAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.KernelFunction.InvokeStreamingAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+MoveNext() at Microsoft.SemanticKernel.KernelFunction.InvokeStreamingAsync[TResult](Kernel kernel, KernelArguments arguments, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource.GetResult() ``` Because the LLM what I used return data like: ```txt data: {"id":"d64e9540-d213-4907-954e-cfc93a840029","choices":[{"delta":{"content":" today","function_call":null,"role":null,"tool_calls":null},"finish_reason":null,"index":0,"logprobs":null}],"created":6030235,"model":"deepseek-chat","object":"chat.completion.chunk","system_fingerprint":null,"usage":null} data: {"id":"d64e9540-d213-4907-954e-cfc93a840029","choices":[{"delta":{"content":"?","function_call":null,"role":null,"tool_calls":null},"finish_reason":null,"index":0,"logprobs":null}],"created":6030235,"model":"deepseek-chat","object":"chat.completion.chunk","system_fingerprint":null,"usage":null} data: {"id":"d64e9540-d213-4907-954e-cfc93a840029","choices":[{"delta":{"content":"","function_call":null,"role":null,"tool_calls":null},"finish_reason":"stop","index":0,"logprobs":null}],"created":6030235,"model":"DeepSeek-LLM-67B-chat","object":"chat.completion.chunk","system_fingerprint":null,"usage":{"prompt_tokens":16,"completion_tokens":10,"total_tokens":26}} data: [DONE] ``` All the previous data can be obtained. Please pay attention to the end mark (`data: [DONE]`) of the last line. This is not a json format. The parsing may return null and an error may be reported when using it later. ## Solution Is it possible to customize the parsing process so that I can perform special processing on the data returned by different language models and finally process it into a unified format?

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.

[markwallace-microsoft]

@RogerBarret0 is this a duplicate of #8407?

[RogerBarret0]

@markwallace-microsoft, this is not. The ask can be addressed with @dmytrostruk's answer in the linked discussion.

We also do not support usage of OpenAI Connector against other models content that do not comply if the OpenAI responses standard, relying on OpenAI SDK internal handling of such scenarios.