confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.2k stars 1.11k forks source link

If you miss SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID in the image configuration you get into big problems #3127

Open orchestr7 opened 4 months ago

orchestr7 commented 4 months ago

In the Schema Registry image configuration, there's a special option called SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID, which sets the Consumer group ID for the Schema Registry in most images.

If you have multiple schema registry instances and forget to set this option, you can encounter significant race condition issues. This is a common problem many people face, as discussed here: https://stackoverflow.com/questions/58563035/multiple-kafka-schema-registry-against-same-cluster

When running a Schema Registry in a multi-node setup, it’s important to configure certain properties to ensure proper communication and coordination between nodes. The SCHEMA_REGISTRY_SCHEMA_REGISTRY_GROUP_ID environment variable is used to set the Kafka Consumer Group ID for the Schema Registry. In a multi-node setup, each instance of Schema Registry should have a unique Consumer Group ID to ensure that they don’t interfere with each other.

Can we make this variable mandatory? Several of my teammates nearly lost their sanity after spending days troubleshooting this silent, flaky behavior.

I suppose it is somewhere near that code: public static final String SCHEMAREGISTRY_GROUP_ID_CONFIG = "schema.registry.group.id";

orchestr7 commented 4 months ago

WDYT? @rayokota