How to set broker config with "real" underscore

[royyeah]

In the upcoming Kafka 2.0.0 (so Confluent 5.0.0-beta1) release it is possible to for instance configure listener.name.sasl_plaintext.plain.sasl.login.callback.handler.class for custom login callback handler (see https://cwiki.apache.org/confluence/display/KAFKA/KIP-86%3A+Configurable+SASL+callback+handlers and https://github.com/apache/kafka/commit/9f8c3167eb2fcab158147eb4fefdabc933b8a3a1)

Am I correct to assume that the Docker environment variable KAFKA_LISTENER_NAME_SASL_PLAINTEXT_PLAIN_SASL_LOGIN_CALLBACK_HANDLER_CLASS will be translated to a config listener.name.sasl.plaintext.plain.sasl.login.callback.handler.class instead of listener.name.sasl_plaintext.plain.sasl.login.callback.handler.class? At least it seams configuration is not properly set in the broker. Or am completely misinterpreting how to use this property?

See also my comment on the related commit for more background: https://github.com/apache/kafka/commit/9f8c3167eb2fcab158147eb4fefdabc933b8a3a1

This is my docker-compose.yml:

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.0.0-beta1-1
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_SASL_ENABLED: "false"

  kafka:
    image: confluentinc/cp-kafka:5.0.0-beta1-1
    depends_on:
      - zookeeper
    volumes:
      - ./security:/etc/kafka/secrets
      - ./jars:/etc/kafka/jars
    ports:
      - "9092:9092"
    environment:
      CLASSPATH: /etc/kafka/jars/*
      ZOOKEEPER_SASL_ENABLED: "false"
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka:9092
      KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
      KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.auth.SimpleAclAuthorizer
      KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "false"
      KAFKA_SUPER_USERS: User:admin
      KAFKA_OPTS: -Djava.security.auth.login.config=/etc/kafka/secrets/broker_jaas.conf
      KAFKA_LISTENER_NAME_SASL_PLAINTEXT_PLAIN_SASL_LOGIN_CALLBACK_HANDLER_CLASS: com.bla.CustomAuthenticateCallbackHandler

[lostick]

We could use update the utility script to allow underscores such as listener.name.sasl_plaintext.plain. One way would be to use double underscores but that a bit ugly IMO

[OneCricketeer]

One possibility, copied from https://hub.docker.com/r/uhopper/hadoop/

• _ => .
• __ => _
• ___ => -

Following are some illustratory examples:

• CORE_CONF_fs_defaultFS: sets the fs.defaultFS property in core-site.xml
• YARN_CONF_yarn_log___aggregation___enable: sets the yarn.log-aggregation-enable property in yarn-site.xml

[dineshudayakumar]

To overcome this property name with "-" issue, I am creating a layer top of the Confluent Kafka docker image which inserts a line to properties file with the properties as needed.

Also, whats the right way to add the jar file containing the callback class. (I am again adding this in the wrapper docker image)

Thank You, Dinesh