Node max concurrent connections

[capablemonkey]

Does http.globalAgent.maxSockets limit the number of requests we can send out at any one time, and thus, change our understanding that all 1000 requests are fired at the same time?

http://stackoverflow.com/questions/12060869/why-is-node-js-only-processing-six-requests-at-a-time

http://nodejs.org/docs/latest/api/http.html#http_agent_maxsockets

Will need to look into this.

[capablemonkey]

Had to hack around. Using a fork of restler which allows you to pass in an https.Agent. Then, using a fork of dwolla-node which uses the forked restler and passes in an Agent with maxSockets = 100.

Results are drastically improved. For 1000 queries to google autocomplete:

Original, using latest restler, default maxSockets on digital ocean box...
Longest time between responses:  142 ms
Shortest time between responses:  1 ms
Average response time interval:  9.474 ms

Shortest response time:  501 ms
Longest response time:  9985 ms
Average response time:  5179.156 ms
==========
Using maxSockets=100 on same box...
Longest time between responses:  4223 ms
Shortest time between responses:  0 ms
Average response time interval:  4.8 ms

Shortest response time:  258 ms
Longest response time:  5117 ms
Average response time:  589.714 ms
==========
using maxSockets=50 on same box...
Longest time between responses:  61 ms
Shortest time between responses:  0 ms
Average response time interval:  0.918 ms

Shortest response time:  231 ms
Longest response time:  1199 ms
Average response time:  688.557 ms

maxSockets=50 has a nearly 10x faster average response time and average response time interval. Woot

[capablemonkey]

Playing with the max sockets limit, it seems more simultaneous sockets is not necessarily beneficial. In fact, too many, it seems, can be very detrimental. To see what I mean, consider the timing results for the default socket limit of 5:

Shortest response time:  556 ms
Longest response time:  9453 ms
Average response time:  4971.681 ms
    ✓ 1000 Account Info requests in quick succession (9459ms)

Then the for a socket limit of 50:

Shortest response time:  655 ms
Longest response time:  7045 ms
Average response time:  4310.631 ms
    ✓ 1000 Account Info requests in quick succession (7051ms)

Socket limit 100:

Shortest response time:  791 ms
Longest response time:  9501 ms
Average response time:  6277.165 ms
    ✓ 1000 Account Info requests in quick succession (9507ms)

Socket limit 500:

Shortest response time:  969 ms
Longest response time:  18349 ms
Average response time:  7671.742 ms
    ✓ 1000 Account Info requests in quick succession (18357ms)

Socket limit 1000:

Shortest response time:  1202 ms
Longest response time:  38369 ms
Average response time:  20442.075 ms
    ✓ 1000 Account Info requests in quick succession (38375ms)

It seems there is a sweet spot where the number of sockets is neither too small, nor too big. If we try 25, this seems to be closer to the sweet spot:

Shortest response time:  590 ms
Longest response time:  6622 ms
Average response time:  3876.045 ms
    ✓ 1000 Account Info requests in quick succession (6628ms)

...which beats 10 sockets by 100ms.

Why is it the case that too many sockets cause considerably slower response times and yet too few will do the same? Does this "sweet spot" change from client machine to client machine?

Or, is it because at higher socket limits, more simultaneous requests are sent to the server and the server struggles to serve them all in a timely manner? But if so, why would the server take longer to serve the same number of requests (1000), if the only variable is the rate at which it receives them?

[capablemonkey]

Fixed this problem in arete, which the project now uses.

Learned that more sockets = more resources. Depending on the resources (and configuration) of the client and server(s), there is an equilibrium where a particular number of concurrent requests will achieve the best responses per second rate.