search

confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.22k stars 1.11k forks source link

How to send jaas config file to schema registry in docker compose through KAFKA_OPTS property? #3215

Open varunkamra opened 2 months ago

varunkamra commented 2 months ago

I am using following docker compose file:

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.7.0
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      #JVMFLAGS: "-Djava.security.auth.login.config=/opt/kafka/config/zoo-jaas.conf"
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      ZOO_PORT: 2181
      KAFKA_OPTS: "-Djava.security.auth.login.config=/opt/kafka/config/zoo-jaas.conf"
    volumes:
      - /opt/kafka/config/zoo-jaas.conf:/opt/kafka/config/zoo-jaas.conf

  broker:
    image: confluentinc/cp-kafka:7.7.0
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "29092:29092"
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_LISTENERS: INTERNAL://:29092,EXTERNAL://:9092
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://broker:29092,EXTERNAL://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:SASL_PLAINTEXT,EXTERNAL:SASL_PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_OPTS: '-Djava.security.auth.login.config=/opt/kafka/config/server-jaas.conf'
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN

    volumes:
      - /opt/kafka/config/server-jaas.conf:/opt/kafka/config/server-jaas.conf
  schema-registry:
    image: confluentinc/cp-schema-registry:7.7.0
    hostname: schema-registry
    container_name: schema-registry
    depends_on:
      - broker
    ports:
      - "8081:8081"
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
      SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081
      SCHEMA_REGISTRY_LOG4J_ROOT_LOGLEVEL: WARN
      SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL: SASL_PLAINTEXT
      SCHEMA_REGISTRY_KAFKASTORE_SASL_MECHANISM: PLAIN
        #SCHEMA_REGISTRY_KAFKASTORE_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="test" password="test" user_admin="test";'  
      SCHEMA_REGISTRY_KAFKA_OPTS: '-Djava.security.auth.login.config=/opt/kafka/config/server-jaas.conf'
      SCHEMA_REGISTRY_LOG4J_LOGGERS: "org.apache.kafka=ERROR,io.confluent.rest.exceptions=FATAL"
    volumes:
      - /opt/kafka/config/server-jaas.conf:/opt/kafka/config/server-jaas.conf

  rest-proxy:
    image: confluentinc/cp-kafka-rest:7.7.0
    depends_on:
      - broker
      - schema-registry
    ports:
      - 8082:8082
    hostname: rest-proxy
    container_name: rest-proxy
    environment:
      KAFKA_REST_HOST_NAME: rest-proxy
      KAFKA_REST_BOOTSTRAP_SERVERS: 'broker:29092'
      KAFKA_REST_LISTENERS: "http://0.0.0.0:8082"
      KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081'
      KAFKA_REST_CLIENT_SECURITY_PROTOCOL: SASL_PLAINTEXT
      KAFKA_OPTS: '-Djava.security.auth.login.config=/opt/kafka/config/server-jaas.conf'
      KAFKA_REST_CLIENT_SASL_MECHANISM: PLAIN
    volumes:
      - /opt/kafka/config/server-jaas.conf:/opt/kafka/config/server-jaas.conf

I am not able to set KAFKA_OPTS for schema registry container, I am required to use SCHEMA_REGISTRY_KAFKASTORE_SASL_JAAS_CONFIG instead which works but I want to provide a conf file instead.

Error:

schema-registry  | [2024-08-09 13:23:31,277] ERROR Error while running kafka-ready. (io.confluent.admin.utils.cli.KafkaReadyCommand)
schema-registry  | org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
schema-registry  |  at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:541)
schema-registry  |  at org.apache.kafka.clients.admin.Admin.create(Admin.java:147)
schema-registry  |  at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
schema-registry  |  at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:136)
schema-registry  |  at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:149)
schema-registry  | Caused by: org.apache.kafka.common.KafkaException: Failed to create new NetworkClient
schema-registry  |  at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:252)
schema-registry  |  at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:189)
schema-registry  |  at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:525)
schema-registry  |  ... 4 more
schema-registry  | Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
schema-registry  |  at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:150)
schema-registry  |  at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:103)
schema-registry  |  at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:87)
schema-registry  |  at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:167)
schema-registry  |  at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
schema-registry  |  at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:119)
schema-registry  |  at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:223)
schema-registry  |  ... 6 more

I found on the confluent documentation that the env variables have to be prefixed with SCHEMA_REGISTRY_ so I have tried with both KAFKA_OPTS and SCHEMA_REGISTRY_KAFKA_OPTS but I keep getting the error.

OS: Ubuntu 24.04

OneCricketeer commented 2 months ago

It's SCHEMA_REGISTRY_OPTS

Source - https://github.com/confluentinc/schema-registry/blob/master/bin/schema-registry-run-class#L88

OneCricketeer commented 2 months ago

confluent documentation that the env variables have to be prefixed with SCHEMAREGISTRY

While correct for this case, that prefix only modifies the server properties file, not the JVM settings. JAVA_TOOL_OPTIONS should work as well since most JVMs support that