search

linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

The follower.replication.throttled.replicas written can be messed up if previous value was '*' #1322

Closed robinp-tw closed 3 years ago

robinp-tw commented 4 years ago

In the setThrottledReplicas function (https://github.com/linkedin/cruise-control/blob/master/cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/executor/ReplicationThrottleHelper.java#L182) we suspect that the joining of the old list and new list with commas can lead to a situation where, if the old value is a single '' which is allowed, then the result is something like `follower.replication.throttled.replicas: ",11:1,11:2....."` which is not a valid value.

Normally Kafka's helper would validate this, but due a bug (https://issues.apache.org/jira/browse/KAFKA-10460) it didn't catch this case.

MatthewBerk commented 3 years ago

Hello, is a fix still needed for this issue?

Also is it just * you are concerned with or are there other characters also?

Lincong commented 3 years ago

Hello, is a fix still needed for this issue?

Also is it just * you are concerned with or are there other characters also?

Hi Matt, I think a fix is still needed. According to the below Kafka ticket, the value must be either * or a list of k:v pairs. In other words, the value cannot be a list of k:v pairs + *.

https://issues.apache.org/jira/browse/KAFKA-10460

I created a PR to fix this issue. https://github.com/linkedin/cruise-control/pull/1381

Feel free to comment on it. Thanks!