Say you have a cluster with 3 nodes, where each node has 1000 upstream connections. If those nodes are becoming overloaded you may increase the number of replicas (either manually or with autoscaling).
Currently, if you add 3 more nodes, you'll end up with 3 nodes having 1000 upstream connections and 3 nodes with 0 upstream connections. Therefore Piko should rebalance upstream connections.
As Piko is designed to be hosted behind a load balancer, if a node drops the connection to an upstream service, that service will reconnect to a random node. Therefore when nodes find they have far more connections than the average for the cluster, then can gradually shed connections to upstreams which will then reconnect to a random node, rebalancing the cluster.
Such as in the above example, the average number of connections across the 6 nodes is 500, but the first three nodes all have 1000 connections each. The threshold and rate of shed connections can be configurable, such as shedding if you have 20% more connections than the cluster average, and shedding 0.5% of connections every second.
Say you have a cluster with 3 nodes, where each node has 1000 upstream connections. If those nodes are becoming overloaded you may increase the number of replicas (either manually or with autoscaling).
Currently, if you add 3 more nodes, you'll end up with 3 nodes having 1000 upstream connections and 3 nodes with 0 upstream connections. Therefore Piko should rebalance upstream connections.
As Piko is designed to be hosted behind a load balancer, if a node drops the connection to an upstream service, that service will reconnect to a random node. Therefore when nodes find they have far more connections than the average for the cluster, then can gradually shed connections to upstreams which will then reconnect to a random node, rebalancing the cluster.
Such as in the above example, the average number of connections across the 6 nodes is 500, but the first three nodes all have 1000 connections each. The threshold and rate of shed connections can be configurable, such as shedding if you have 20% more connections than the cluster average, and shedding 0.5% of connections every second.