Configurable server throttle

salesforce / grpc-java-contrib

Useful extensions for the grpc-java library

BSD 3-Clause "New" or "Revised" License

220 stars 34 forks source link

Configurable server throttle #36

Closed rmichela closed 5 years ago

rmichela commented 6 years ago

Out of the box, gRPC will use unlimited resources. By default it uses an executor with an unbounded work queue and an unlimited number of worker threads.

A configurable server throttle would allow you to constrain these dimensions for a server.

Set a server's max concurrency
Set the max amount or waiting work before turning away requests

If a request gets turned away because of too much work, reply with a UNAVAILABLE status.

rmichela commented 6 years ago

See: https://groups.google.com/forum/#!topic/grpc-io/LrnAbWFozb0

rmichela commented 6 years ago

Netflix did it!

https://medium.com/@NetflixTechBlog/performance-under-load-3e6fa9a60581
https://github.com/Netflix/concurrency-limits