search

grpc / grpc-java

The Java gRPC implementation. HTTP/2 based RPC
https://grpc.io/docs/languages/java/
Apache License 2.0
11.4k stars 3.83k forks source link

Provide channel pooling #2370

Open garrettjonesgoogle opened 7 years ago

garrettjonesgoogle commented 7 years ago

The client libraries generated by https://github.com/googleapis/toolkit currently create a new channel for every API wrapper object, which is expensive and wasteful. For the benefit of users, we need channel pooling support so that startup time and resource usage is minimized, regardless of how many API wrapper objects are created. The alternatives require more work for users (which is undesirable) or implementing pooling support in the API wrapper layer, which doesn't make sense if grpc is a better place to implement it.

ejona86 commented 7 years ago

Channels are lazily connected. So creating a channel is cheap, but then the connection it has to make on first use is expensive. But creating many channels on startup shouldn't be too prohibitive by itself.

Channels support auto-IDLEness. You can configure a timeout where after no active RPCs the channel will auto-disconnect. We also have a connection state API (although it isn't yet plumbed) that can be notified when the channel goes IDLE. Thus, with auto-IDLE you could easily manage the cleanup of unused Channels by shutting them down after IDLE. There's races involved and details, but wanted to mention it.

  1. C core currently auto-shares transports across channels. We could do the same. This would mean if you have many long-lived channels all to the same host and transport configuration, they automatically share. It does not work well for use-it-once style of work which would be most similar to what people normally expect from HTTP clients.
  2. Another option is to have a "pool" object that you request channels from (probably with ref counting, so you have to return items to the pool). It's unclear whether this would be global. Global provides the best gains, especially when different code is mixed, but is hard from a configuration standpoint. Note that configuration here implies all configuration of the Channel, not just transport configuration. Most importantly, that includes the default executor used. This solution would be best for users expecting to interact like they do with HTTP clients today.
  3. We could also have both. That might be the best of both worlds, but not sure it is worth the effort or just split the user base based on which model they prefer.
gravetii commented 7 years ago

@ejona86 I can see that this is added under the Next milestone. Till then, what would be the suggested way of pooling ManagedChannel objects in a grpc client? Also, is it right when I say that a single channel has multiple TCP connections to the server?

lukaszx0 commented 7 years ago

Also, is it right when I say that a single channel has multiple TCP connections to the server?

Yes, it is possible to have ManagedChannel backed by multiple connections to one or many servers if you use ManagedChannel2 and RoundRobinLoadBalancer2 or have a custom LoadBalancer2 implementation, one which creates multiple subchannels (the 2 is a new next version with revamped API)

ejona86 commented 7 years ago

@sandeepd-bsb, I'd suggest rolling your own. It's unclear what you need in way of "pooling". For single-host use-cases I'd suggest to just have a static ManagedChannel with auto-idle enabled. If you are communicating with an unbound number of hosts, then you'll need to figure out something yourself.