Topic creation

[va77]

Hello everyone, I have a question related to topic creation. I have 2 racks, each have 2 brokers, so total brokers of 4. I created a topic using the following command: kafka-topic.sh —topic test — create —partitions 8 —replication-factor 4 and topic was successfully created. At the same time I tried to create a topic through topicctl: partitions: 8 replicationFactor: 4 placement strategy: in-rack

but I got an error: Rack does not have enough brokers for in-rack placement. After the error topicctl stopped and didn’t create other topics, but after the error “test” appeared according to the kafka-topic.sh.

then I changed replicationFactor from 4 to 2 and after that all 15 topics successfully created.

Could someone explain why it was created by kafka-topic.sh and wasn’t created by topicctl?

[erushing]

kafka-topic.sh will not place in-rack, so it can place your partition across all 4 brokers.

In-rack placement means the leader of the partition and all of its replicas are placed in the same rack (zone). If you have a replication factor of 4, but only 2 brokers in each rack/zone, topicctl cannot place everything because you only have effectively 2 brokers for each partition. It places a leader on 1 broker, a replica on another and then has 2 more replicas it cannot place because topicctl does not want to place more than one replica on a single broker (because that would not achieve higher availability for this partition). You must either decrease replication factor to 2, increase brokers in each zone to 4 or don't use in-rack placement.