Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
In our Kafka clusters, there are peak and off-peak hours. During peak hours the loads are high but it's low during off-peak. Loads especially CPU usage skews a lot (STD is high) during peak hours so we need rebalancing.
But because the loads are high, we try running rebalance during off-peak hours to avoid affecting peak hour performance. But the loads is balanced only then, when peak hours come, it becomes skew again. So this is not effective.
As our clusters don't have changes like topic creation, increase partition... too frequently, I think it's good to have an option for us to generate proposal plans during peak hours, and apply it later during off-peak hours.
Please consider reviewing this feature.
Thank you !
mhratson
commented
1 month ago
In our Kafka clusters, there are peak and off-peak hours. During peak hours the loads are high but it's low during off-peak. Loads especially CPU usage skews a lot (STD is high) during peak hours so we need rebalancing. But because the loads are high, we try running rebalance during off-peak hours to avoid affecting peak hour performance. But the loads is balanced only then, when peak hours come, it becomes skew again. So this is not effective. As our clusters don't have changes like topic creation, increase partition... too frequently, I think it's good to have an option for us to generate proposal plans during peak hours, and apply it later during off-peak hours. Please consider reviewing this feature. Thank you !