While trying to tackle the issue of OOM errors from having unbounded outstanding requests in asynchronous servers, I came across this SO answer and this presentation.

Each commit in this branch addresses a particular slide in the presentation:

Add load control handler - slide 33 (this is the main change)
Set child WRITE_BUFFER_WATER_MARK - slide 11
Use voidPromise on unused future - slide 8
Send responses inside the event loop - slide 28
Write to context to continue pipeline - slide 29

PingPongServerBenchmark

No changes

Benchmark                                      Mode  Cnt       Score      Error  Units
PingPongServerBenchmark.benchmark             thrpt   50  121435.443 ± 5985.188  ops/s
PingPongServerBenchmark.benchmark:actualQPS   thrpt   50  121456.863 ± 6018.500  ops/s
PingPongServerBenchmark.benchmark:busyQPS     thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:errorQPS    thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:networkQPS  thrpt   50     190.500 ±  666.834  ops/s
PingPongServerBenchmark.benchmark:timeoutQPS  thrpt   50         ≈ 0             ops/s

Changes (load control = off)

Benchmark                                      Mode  Cnt       Score      Error  Units
PingPongServerBenchmark.benchmark             thrpt   50  132054.557 ± 6448.650  ops/s
PingPongServerBenchmark.benchmark:actualQPS   thrpt   50  132055.211 ± 6292.013  ops/s
PingPongServerBenchmark.benchmark:busyQPS     thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:errorQPS    thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:networkQPS  thrpt   50     189.710 ±  664.069  ops/s
PingPongServerBenchmark.benchmark:timeoutQPS  thrpt   50         ≈ 0             ops/s

Changes (load control = on)

Benchmark                                      Mode  Cnt       Score      Error  Units
PingPongServerBenchmark.benchmark             thrpt   50  128227.967 ± 6005.391  ops/s
PingPongServerBenchmark.benchmark:actualQPS   thrpt   50  128197.647 ± 6459.743  ops/s
PingPongServerBenchmark.benchmark:busyQPS     thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:errorQPS    thrpt   50         ≈ 0             ops/s
PingPongServerBenchmark.benchmark:networkQPS  thrpt   50     198.261 ±  694.001  ops/s
PingPongServerBenchmark.benchmark:timeoutQPS  thrpt   50         ≈ 0             ops/s

Codecov Report

Merging #203 into master will increase coverage by 0.47%. The diff coverage is 89.19%.

@@             Coverage Diff              @@
##             master     #203      +/-   ##
============================================
+ Coverage      73.4%   73.87%   +0.47%     
  Complexity       10       10              
============================================
  Files            87       88       +1     
  Lines          2767     2801      +34     
  Branches        348      355       +7     
============================================
+ Hits           2031     2069      +38     
+ Misses          510      506       -4     
  Partials        226      226

Impacted Files	Coverage Δ	Complexity Δ
...java/com/uber/tchannel/handlers/RequestRouter.java	`76.58% <100%> (+2.26%)`	`0 <0> (ø)`	:arrow_down:
.../src/main/java/com/uber/tchannel/api/TChannel.java	`68.12% <75%> (+0.21%)`	`0 <0> (ø)`	:arrow_down:
...com/uber/tchannel/handlers/LoadControlHandler.java	`89.29% <89.29%> (ø)`	`0 <0> (?)`
...e/src/main/java/com/uber/tchannel/api/TFuture.java	`65.85% <0%> (+4.88%)`	`0% <0%> (ø)`	:arrow_down:
...n/java/com/uber/tchannel/messages/RawResponse.java	`50% <0%> (+8.7%)`	`0% <0%> (ø)`	:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 18b6586...45483d4. Read the comment docs.

uber / tchannel-java

Implement server backpressure (opt-in) #203

PingPongServerBenchmark

Codecov Report