search

confluentinc / cp-docker-images

[DEPRECATED] Docker images for Confluent Platform.
Apache License 2.0
1.14k stars 703 forks source link

Setting SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR #448

Open zoltan-fedor opened 6 years ago

zoltan-fedor commented 6 years ago

Settings SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR doesn't work:

Dockerfile:

  schema_registry:
    image: confluentinc/cp-schema-registry:4.0.0-3
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR: 3

Container log: WARN The configuration 'topic.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)

The kafkastore.topic.replication.factor is an existing config value, see https://docs.confluent.io/current/schema-registry/docs/config.html

The same works for some other config values, like SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS

mamoit commented 5 years ago

I can verify this, I get exactly the same, and then:

WARN Creating the schema topic _schemas using a replication factor of 1, which is less than the desired one of 3. If this is a production environment, it's crucial to add more brokers and increase the replication factor of the topic. (io.confluent.kafka.schemaregistry.storage.KafkaStore)

@zoltan-fedor did you figure out what it was?

elvis-cai commented 5 years ago

same here if I set SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR to 3, The configuration 'topic.replication.factor' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) for 5.2.1

robcowart commented 5 years ago

This works for me in CP 5.3.1. It might have worked in earlier versions as well.

From my docker-compose file...

# kafkastore.topic.replication.factor
      #   The desired replication factor of the schema topic. The actual replication factor will be the smaller of this
      #   value and the number of live Kafka brokers.
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC_REPLICATION_FACTOR: 2

The result is that the topic has 2 replicas.

I think this can be closed.

OneCricketeer commented 3 years ago

Alright, well, glad it worked for you.

Here's the source of truth, which I believe the documentation is generated based on

https://github.com/confluentinc/schema-registry/blob/master/core/src/main/java/io/confluent/kafka/schemaregistry/rest/SchemaRegistryConfig.java#L87