search

salesforce / reactive-grpc

Reactive stubs for gRPC
BSD 3-Clause "New" or "Revised" License
833 stars 118 forks source link

How do you configure how much data is buffered on the producer side? #255

Open bbemis017 opened 3 years ago

bbemis017 commented 3 years ago

I've been poking around different parts the gRPC documentation and I am not sure if I am understanding where data gets buffered correctly.

I have noticed that at the gRPC layer there is a method called .flowControlWindow(int capacity);. It seems like you can set a capacity on the consumer side and this does seem to impact when back pressure kicks in on the server (producer) side. So I think this is controlling the size of the consumer side buffer.

So I was thinking that if I set the flowControlWindow on the server side (producer) from the NettyServerBuilder I should be able to control how much data get's buffered on the server when back pressure kicks in, but it doesn't seem to have that effect.

Is that because the the flowControlWindow does not exist on the producer? This diagram shows flow control windows on both sides: image

Is there a way in reactive-grpc to control how much data is buffered on the server (producer) side?

bbemis017 commented 3 years ago

I think I'm understanding this now. For anyone else with this question I found the diagram above to be a little misleading when you are thinking about buffers. If I could modify the bottom part I would draw something more like this:

image

A key difference to understand here is that flowControlWindows are only defined on the consuming end of gRPC. So if you set the flowControlWindow on the NettyChannelBuilder it is referring to the flowControlWindow on the client side where the client is the consumer. And if you set the flowControlWindow on the NettyServerBuilder it is referring to the flowControlWindow on the server side where the server is the consumer. I really wish they mention that in the documentation for those methods lol.

There is in fact an unbounded buffer on the producer side of gRPC. I'm referring to it in that diagram as the pendingWriteQueue because that's what it is called in netty. The gRPC onReady() method notifies you whenever the size of that buffer drops below the ONREADY_THRESHOLD (32kb).

This is where configuring buffers get's a little frustrating because gRPC has not yet decided how they want the ONREADY_THRESHOLD to be configured:

So currently we are stuck with a threshold of 32kb. Since reactive-grpc adheres to the onReady threshold reactive-grpc also locks you into this threshold.

I am wondering if reactive-grpc/rxJava should provide some mechanism for setting your own bufferSize/onReadyThreshold so that you can work with a larger buffer if you need to. Currently I'm not sure if there is a way to workaround it if you are using reactive-grpc but I could be wrong.

The reason why I'm digging into this so much is because it seems like buffer sizes can have an impact on performance or the throughput of your gRPC stream. I'm not sure if the current buffer size is an issue for me yet but I'm hoping to do some performance testing in the next couple months to find out and I think it would be very helpful to be able to modify the buffer size.