Adding bulk metric send

[beevee]

This code sends all performance monitor counters in one request, separated with newlines. Saves a lot of open sockets for carbon-cache, if you have a lot of metrics.

[tas50]

This would be really great to get merged in. In large deployments the sockets getting rapidly opened actually causes carbon-cache to tip over.

[MattHodge]

@beevee thanks for this looks great. Will test it in next few days and merge.

@tas50 It will :) been traveling for last 2 months but back home tomorrow..

[tas50]

I thought I'd pass along some stats. I had 22 windows servers and about 500 Linux systems and with the Windows systems added to Graphite I completely tipped over carbon cache. I thought it was my old server so I swapped out for a brand new 20 core server with 12 disks, but the system still tipped over. I rolled out the bulk update code and went from a 15 second period to a 30 second period and here's what's happening now:

Packets received on my graphite server:

Memory usage:

Load:

Update time:

So bottom line: @beevee saved my Graphite server. Thanks

[beevee]

@tas50 you are welcome!

Please, make sure that all your metrics are correctly sent and none are lost. I don't have any production systems to check, only a weak test machine. Which is why I noticed this ineffective behavior in the first place :)

[MattHodge]

@tas50 I had the same performance improvement in my environment! Thank you :)

Carbon CPU Usage after making the changes on all servers in my environment: