linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

Skip generating metric sample for broker 2 because all broker metrics are missing - Waning logs reported from cruisecontrol #2205

Open nsettipa opened 4 days ago

nsettipa commented 4 days ago

Hi,

We have Strimzi Kafka setup with 3 cluster broker pods and cruise-control to monitor the metrics. In Cruise-control, below warning is reported:

Skip generating metric sample for broker 2 because all broker metrics are missing. This warning is reported randomly during traffic scenario. In Kafka Broker Pods, there is already Cruise-control metrics jar installed. when this warning reported. How to suppress this warning message.

skipped by broker {2=6353}) partition metric samples and 2(1 skipped) broker metric samples for timestamp 1726200415890.. I verified .

In SamplingUtils.Java, below warning reported when brokerload is NULL private static boolean skipBuildingBrokerMetricSample(BrokerLoad brokerLoad, int brokerId) { if (brokerLoad == null) { LOG.warn("{}broker {} because all broker metrics are missing.", SKIP_BUILDING_SAMPLE_PREFIX, brokerId); return true;

When this warning reported and how to resolve this issue. would require technical support.

Thank you.

Rgds, Narasimham

nsettipa commented 10 hours ago

Any update on the ticket