search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

Kafka Cruise Control - NumValidWindows: (0/0) (NaN%), NumValidPartitions: 0/844 (0.000%) #237

Closed cnutext closed 6 years ago

cnutext commented 6 years ago

Facing issues setting up kafka cruise control with kafka cluster with 3 brokers.

I was trying to set-up cruise control localhost(locally running kafka broker and cruise control) and was successful: MonitorState: {state: RUNNING(1.000% trained), NumValidWindows: (1/1) (100.000%), NumValidPartitions: 138/138 (100.000%), flawedPartitions: 0}...}

But the issue is , when I started setting up on 3 broker Kafka 1.1 cluster, I am not seeing any progress: {MonitorState: {state: RUNNING(0.000% trained), NumValidWindows: (0/0) (NaN%), NumValidPartitions: 0/844 (0.000%), flawedPartitions: 0}, ExecutorState: {state: NO_TASK_IN_PROGRESS, finished/total leadership movements: 0/0}, AnalyzerState: {isProposalReady: false, ReadyGoals: []}}

Can some please help on this, if I am missing any configs here? in CC logs I am seeing: [2018-05-23 15:27:51,733] INFO Collected 0 partition metric samples for 0 partitions. Total partition assigned: 844. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher) [2018-05-23 15:27:51,733] INFO Collected 0 broker metric samples for 0 brokers. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher) [2018-05-23 15:27:51,733] INFO Finished sampling in 5147 ms. (com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager) [2018-05-23 15:27:53,437] INFO Skipping best proposal precomputing because load monitor does not have enough snapshots. (com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer) [2018-05-23 15:28:23,438] INFO Skipping best proposal precomputing because load monitor does not have enough snapshots. (com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer) [2018-05-23 15:28:53,438] INFO Skipping best proposal precomputing because load monitor does not have enough snapshots. (com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer) [2018-05-23 15:29:23,438] INFO Skipping best proposal precomputing because load monitor does not have enough snapshots. (com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer)

cnutext commented 6 years ago

Closed the issue because, it was configuration fix. I have added below configuration into the Kafka server:

########### server.properties:cruise.control.metrics.reporter.bootstrap.servers=0.0.0.0:9092 ###########

That did the magic: {MonitorState: {state: RUNNING(20.000% trained), NumValidWindows: (1/1) (100.000%), NumValidPartitions: 124/124 (100.000%), flawedPartitions: 0}, ExecutorState: {state: NO_TASK_IN_PROGRESS, finished/total leadership movements: 0/0}, AnalyzerState: {isProposalReady: true, ReadyGoals: [NetworkInboundUsageDistributionGoal, CpuUsageDistributionGoal, PotentialNwOutGoal, NetworkInboundCapacityGoal, LeaderBytesInDistributionGoal, DiskCapacityGoal, ReplicaDistributionGoal, RackAwareGoal, TopicReplicaDistributionGoal, NetworkOutboundCapacityGoal, CpuCapacityGoal, DiskUsageDistributionGoal, NetworkOutboundUsageDistributionGoal, ReplicaCapacityGoal]}}

Reason: it was due to kafka was running on host IP not on localhost , above config helped to fix it