Closed MoienTajik closed 5 days ago
@annelo-msft, we need allow setting/calling HttpRequestMessage.SetBrowserResponseStreamingEnabled(true) on HttpClient's messages to fix this problem.
Tracking with https://github.com/Azure/azure-sdk-for-net/issues/44706
@KrzysztofCwalina, I believe we may already have the method we need on HttpClientPipelineTransport in the OnSendingRequest method.
For example, following the repro steps above, I am able to create a sample project and add a sample transport implementation that overrides OnSendingRequest:
public class BlazorHttpClientTransport : HttpClientPipelineTransport
{
protected override void OnSendingRequest(PipelineMessage message, HttpRequestMessage httpRequest)
{
httpRequest.SetBrowserResponseStreamingEnabled(true);
}
}
The default SCM transport already passes HttpCompletionOption.ResponseHeadersRead
to HttpClient.Send, so I believe that requirement should be addressed without any additional customization to the transport implementation.
That can then be added to the client by passing an instance of OpenAIClientOptions as follows:
OpenAIClientOptions options = new();
options.Transport = new BlazorHttpClientTransport();
ChatClient client = new(model: "gpt-4o", apiKey, options);
If this doesn't address the issue as you were thinking @KrzysztofCwalina, let me know.
Or @MoienTajik, if this doesn't address the problem you're seeing, I'm happy to dig further into needs here.
Thanks!
Thanks, Anne, for investigating this! I can confirm that this works and solves the problem.
There is a problem with streaming using
IAsyncEnumerable
in Blazor WASM. When consuming the stream in a console app or any environment other than Blazor WASM, the items are processed one by one as they are fetched. However, in Blazor WASM, the stream behaves differently, and the response is only available after all items are fetched from the remote resource.The issue affects both
ChatClient.CompleteChatStreamingAsync
andAssistantClient.CreateRunStreamingAsync
methods (and potentially any other streaming methods).Steps to reproduce:
Code:
Expected Behavior:
Each item should be processed as it is fetched from the remote resource, similar to the behavior observed in a console app.
Actual Behavior:
In Blazor WASM, the stream processes all items only after they are completely fetched, rather than one by one.
Environment:
Additional Information:
The problem is detailed in this blog post. The behavior discrepancy between Blazor WASM and other environments needs to be addressed to ensure consistent streaming functionality by setting
SetBrowserResponseStreamingEnabled(true)
andHttpCompletionOption.ResponseHeadersRead
on theHttpRequestMessage
.