linkedin / cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
https://github.com/linkedin/cruise-control/tags
BSD 2-Clause "Simplified" License
2.74k stars 587 forks source link

How to change strimzi cruise control default topic replication factor to 3? #1942

Closed bharat15-ml closed 1 year ago

bharat15-ml commented 1 year ago

I need to increase the Strimzi cruise control default topic (modeltrainingsamples, partitionmetricsamples) replication factor to 3, tried it below configurations, but it doesn't work.

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.3.1
    replicas: 3
    listeners:
    - name: plain
      port: 9092
      type: internal
      tls: false  
    - name: tls
      port: 9093
      type: internal
      tls: true
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.3"
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
  topicOperator: {}
  userOperator: {}
  cruiseControl:
    config:
      cruise.control.modeltrainingsamples.topic: strimzi.cruisecontrol.modeltraingsamples   
      cruise.control.modeltraingsamples.topic.replication.factor: 3
      cruise.control.modeltrainingsamples.topic.min.insync.replicas: 2
      cruise.control.modeltrainingsamples.topic: strimzi.cruisecontrol.partitionmetricsamples
      cruise.control.partitionmetricsamples.topic.replication.factor: 3
      cruise.control.partitionmetricsamples.topic.min.insync.replicas: 2

Could you suggest changing the default topic replication factor of strimzi cruise control topics?

bharat15-ml commented 1 year ago

For changing the default replication factor of the sample store topics [1], partition.metric.sample.store.topic and broker.metric.sample.store.topic, configure the Cruise Control config section of the Kafka resource like this:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
   name: my-cluster
spec:
...
cruiseControl:
  config: 
    sample.store.topic.replication.factor: 3