Currently the rebalancer does not care about max sending and max receiving buckets in parallel on the storages. It only calculates perfect balance and tells who and to whom should send how many buckets.
This becomes a problem when cluster is huge (100 replicasets, for example), one new is added, and it has max receiving, say, < 10. To prevent high CPU usage. Then the replicasets, which won't fit into the quota, will spam log messages about throttling.
Instead, the rebalancer could see the sending and receiving quotas in advance to plan the max capacity of the incoming channel to the new replicaset and how to utilize it better. Then the throttling wouldn't be not needed almost always.
Currently the rebalancer does not care about max sending and max receiving buckets in parallel on the storages. It only calculates perfect balance and tells who and to whom should send how many buckets.
This becomes a problem when cluster is huge (100 replicasets, for example), one new is added, and it has max receiving, say, < 10. To prevent high CPU usage. Then the replicasets, which won't fit into the quota, will spam log messages about throttling.
Instead, the rebalancer could see the sending and receiving quotas in advance to plan the max capacity of the incoming channel to the new replicaset and how to utilize it better. Then the throttling wouldn't be not needed almost always.