grobian / carbon-c-relay

Enhanced C implementation of Carbon relay, aggregator and rewriter
Apache License 2.0
380 stars 107 forks source link

Canarying carbon-c-relay #223

Closed jjneely closed 5 months ago

jjneely commented 8 years ago

I'm building a new Graphite cluster and would like to ramp up the metrics it ingests. The plan was to have my carbon-c-relay tier send about 20% (or otherwise partial) stream to the new cluster while continuing to send all metrics to the production cluster. Ideally, this X% would be based off the hash so I always get the same metrics routed to the new cluster. As I test that the new cluster scales appropriately I can raise the fraction of metrics sent to it until it reaches 100%.

Perhaps this is a feature request, I don't see how this is currently possible.

grobian commented 7 years ago

I have a random idea here, no idea if it works:

cluster foo fnv1a_ch
   ip-of-target-server
   127.0.0.127:12345
   127.0.0.128:12345
   127.0.0.129:12345
   127.0.0.130:12345
;

Now ip-of-target-server should receive ~20% of traffic, while the queues for port 12345 fill up and start dropping traffic. If you don't want to see this, nc -l -k 127.0.0.1 12345 > /dev/null should work very well to consume any traffic very quickly ;)

grobian commented 7 years ago

you could even use the aliasing system to work up your percentages more easily like

  ip=a
  ip=b
  127.0.0.1:12345=c
  127.0.0.1:12345=d
  127.0.0.1:12345=e
grobian commented 5 months ago

real ramping up or mirroring a percentage etc. is not supported, and I don't have time/resources to implement it either, so one has to do with tricks as suggested in this issue