search

Yelp / kafka-utils

Apache License 2.0
313 stars 127 forks source link

Add support for removing brokers in replace #248

Closed Baisang closed 4 years ago

Baisang commented 4 years ago

This adds support for removing brokers from replica assignment. We specify a replica removal by having a source-broker with no dest-broker. Replica addition could also be added in a similar way, but I'm gonna leave that for the next PR.

Before it was not possible because the script would do two steps of verification for replacement:

  1. Check that the overall replication factor of the topic partitions has not changed. This check can now be skipped with the --rf-changes flag
  2. Check that the replication factor of topic partitions within the same topic is the same. This check can now be skipped with the --rf-mismatch flag.

We also introduce a --topic-partition-filter flag, that is a path to some output from kafka-check, so that you can focus on fixing offline or specific partitions instead of generating a plan for all of them.

Wrote some unit tests and will be trying this out in dev later today, just want to get the review out first.

A sample invocation to generate a reassignment plan to remove a faulty broker id 0 might be:

kafka-cluster-manager -t test_cluster --write-to-file reassign_plan.json replace-broker --source-broker 0 --max-leader-changes 5000 --max-partition-movements 5000 --topic-partition-filter topic_partitions.txt --rf-mismatch --rf-change

where topic_partitions.txt is similar to kafka-check output:

my_topic:0
other_topic:1

While the above command is rather long, it should be much better than relying on adhoc scripts to generate reassignment json files.