search

grpc / grpc-dotnet

gRPC for .NET
Apache License 2.0
4.22k stars 776 forks source link

gRPC Streaming From C# Resulting in GoAway error from server #2358

Open psbang opened 10 months ago

psbang commented 10 months ago

In the solution attached below, we're trying to make gRPC requests to the Google GenerativeAI API, which is supposed to return a streaming response. However, this is inconsistently successful, as a significant fraction of the streaming responses are cut short with a GoAway error (seen in the txt output file also attached below). In the output, you can see the GoAway comes with output frame 13, but we have noticed that the frame number at which this happens, if at all, is also not consistent.

To reproduce our issue, install open the console app included in the attached ZIP file. You will need an API key from https://ai.google.dev/tutorials/setup and fill it in where the // TODO is. Then, run the console app. The app will make 100 calls to the generative AI API with a pre-filled prompt, and at the end you will see how many calls failed (ranges from anywhere between 1 failure to up to 26). I create the proto file using the structures I need from the publicly available protobuf here.

The gRPC calls to this server are consistently successful from the Python client, but we haven't gotten it to succeed 100% of the time with C#. We believe that there might be a flag or header that we're missing to prevent the server from sending us the GoAway, but we don't know what it could be. Please advise.

GrpcStream.zip-20240111T225040Z-001.zip SampleGrpcStreamOutput.txt

erdalsivri commented 10 months ago

I should mention that we tried passing a custom SocketsHttpHandler with various Keepalive values (especially KeepAlivePingPolicy.WithActiveRequests) but still couldn't get it to a reliable state. It is a fair point to say the server has a bug but GenerativeAI API team points us to the working Python client to prove the server is working fine. We did test the python client with a similar loop and it's never failed as consistently as the C# client. So we are thinking there might be some configuration mismatch between the client and the server or a misbehavior on the client side that cause server to reject the request after some time.

JamesNK commented 10 months ago

Hi

This is the same problem as https://github.com/grpc/grpc-dotnet/issues/2361. You can read that issue for more details.

Adding this code to the start of the client app disables RTT pings fixes the problem:

AppContext.SetSwitch("System.Net.SocketsHttpHandler.Http2FlowControl.DisableDynamicWindowSizing", true);
karelz commented 9 months ago

@psbang are you using .NET 6.0 or would a fix in .NET 8.0 be sufficient for you?