search

grobian / carbon-c-relay

Enhanced C implementation of Carbon relay, aggregator and rewriter
Apache License 2.0
380 stars 107 forks source link

Impossible to use any_of on a cluster of nodes #423

Closed Antoine-Auffret closed 2 years ago

Antoine-Auffret commented 3 years ago

Hello, I'm trying to configure carbon-c-relay to load balance (sharding) half the traffic to an group of nodes (node-1-replication) and the other half on another group of nodes (node-2-replication). Each group of nodes have the same data, there are replicated. But carbon-c-realy can't use any_of on a group of nodes. See below for the detail configuration. 

cluster node-1-replication
  forward
    node-1-1:2003
    node-1-2:2003;

cluster node-2-replication
  forward
    node-2-1:2003
    node-2-2:2003;

cluster sharding
  any_of
    node-1-replication
    node-2-replication;

match *
  send to sharding;

Can this architecture works, if yes with what configuration ? Thanks for your support.

grobian commented 3 years ago

So, you want each metric to end up at any set of two stores, but you don't care which one.

Normally, people would use a consistent hash for this, but I guess you want two nodes that are identical, or something.

clusters cannot target other clusters, so what you could do to mimick this is to perform the double send from the matchrule instead:

cluster side-a fnv1a_ch
  node-1-1:2003=somethingrandomA
  node-2-1:2003=whateverrandomB
;
cluster side-b fnv1a_ch
  node-1-2:2003=somethingrandomA
  node-2-2:2003=whateverrandomB
;
match * send to side-a side-b;

because both clusters have the same input to the consistent-hash-ring, they will make the same decision on whether it is node 1 or 2, e.g. both node-1-1 and node-1-2 receive an identical set of metrics, as in your example above.

grobian commented 2 years ago

chaining of clusters is not something I'd like to support at this point