During stress testing, we discovered that the memory profile of opening many connections and executing a single non-streaming RPC call results in strange memory behavior, especially relative to executing many non-streaming calls on a single connection. In particular, we see some closures relating to HPACK slowly pile up before reaching a constant heap residency of anywhere between 2M and 18M. (Exactly what this heap residency is depends on the RTS sample rate, strangely, but that's another can of worms that grapesy probably doesn't need to care about.)
Each of the lines in the above chart marks 1000 new connections.
Thankfully, the memory complexity still appears constant, but this is nonetheless worth investigating to see if we can reduce the residency.
Contrast this against doing the same number of calls on a single connection:
Much lower and more consistent residency.
Reproduce
To reproduce, run the test-stress executable server and client:
During stress testing, we discovered that the memory profile of opening many connections and executing a single non-streaming RPC call results in strange memory behavior, especially relative to executing many non-streaming calls on a single connection. In particular, we see some closures relating to HPACK slowly pile up before reaching a constant heap residency of anywhere between 2M and 18M. (Exactly what this heap residency is depends on the RTS sample rate, strangely, but that's another can of worms that grapesy probably doesn't need to care about.)
Each of the lines in the above chart marks 1000 new connections.
Thankfully, the memory complexity still appears constant, but this is nonetheless worth investigating to see if we can reduce the residency.
Contrast this against doing the same number of calls on a single connection:
Much lower and more consistent residency.
Reproduce
To reproduce, run the
test-stressexecutable server and client:Don't forget to enable
-finfo-table-map -fdistinct-constructor-tableson at leastgrapesyand probablyhttp2.