search

tkestack / kvass

Kvass is a Prometheus horizontal auto-scaling solution , which uses Sidecar to generate special config file only containes part of targets assigned from Coordinator for every Prometheus shard.
Apache License 2.0
614 stars 90 forks source link

Feature/target rebalance #69

Open GoneLikeAir opened 3 years ago

GoneLikeAir commented 3 years ago

Hi, recently I have finish a feature: target rebalance. It can make the load of prometheus more balance.

Why:

  1. The access frequency of deferent job is not the same;
  2. The series of targets in one group are close;
  3. If most of the targets in one group assign to one shard, the load of prometheus may not balance.

How:

  1. Group the target by job;
  2. Check the distribution status of the targets, judge whether it is balanced(check the diff between Max and Min shard);
  3. If not, move target from max shard to min shard.

This is the effect of this feature: Before: image image

After: image

image