search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.77k stars 595 forks source link

Throttling intra broker rebalances #2199

Open sszp opened 2 months ago

sszp commented 2 months ago

Newer versions of Kafka (for example 3.6) supports the dynamic configuration parameter replica.alter.log.dirs.io.max.bytes.per.second to throttle replica movements between log directories within the same broker. This throttling is not available in Cruise Control at the moment, this can cause performance issues for Kafka clients while such reassignment is running. Please add a configuration to Cruise Control that can be used to control this type of throttling as well.

cpaika commented 2 months ago

There's a PR open for this: https://github.com/linkedin/cruise-control/pull/2145