Fix replication factor

[blootsvoets]

The variables for kafka-init in docker-compose were not the ones used in the Dockerfile. Changed them to their actual values.

This will not affect existing topics though. To change the replication factor of existing topics, based on a comment from Stackoverflow, the following should work:

docker-compose exec kafka-1 /bin/bash

function expandTopicsFrom2To3() {
  TOPIC=$1
  echo '{"version":1,"partitions":[{"topic":"'${TOPIC}'","partition":0,"replicas":[1,2,3]},{"topic":"'${TOPIC}'","partition":2,"replicas":[1,2,3]},{"topic":"'${TOPIC}'","partition":1,"replicas":[1,2,3]}]}' > /tmp/topics-to-expand.json
  kafka-reassign-partitions --zookeeper zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181 --execute --reassignment-json-file /tmp/topics-to-expand.json
  kafka-topics --zookeeper zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181 --describe --topic ${TOPIC}
}

expandTopicsFrom2To3 questionnaire_esm

[afolarin]

I'll discuss with @yatharthranjan but generally I would have thought 3x makes sense (especially for those low volume topics).

[yatharthranjan]

@afolarin and @blootsvoets Sounds good to me. FYI you can also generate and reassign partition assignments using the kafka-manager UI.

Also in the above it maybe worth to change the order of replicas for each partition. Eg for partition 1 it is [1,2,3], then for partition 2 it can be [2,1,3] and for partition 3 can be [3,2,1] and also to run preferred-leader-election after the reassignment.

[blootsvoets]

Running it through the Kafka manager makes sense. @yatharthranjan could you please take a look if this can be approved?