SASL/Plain authentication from Schema Registry to Kafka cluster

[vascop]

Is there support for setting up the Schema Registry with SASL/PLAIN? I'm running no SSL between nodes, just SASL/PLAIN from Kafka clients to Kafka brokers (Zookeeper has no SASL configured).

I can connect regular kafka producers/consumers by setting sasl mechanism and security protocol in .properties files and setting the KAFKA_OPTS env variable with -Djava.security.auth.login.config=/etc/schema-registry/kafka_client_jaas.conf

So that setup works. When introducing the schema registry into this though, I realize that there's no fixture like the cluster-bridged-sasl.yml for kafka, just cluster-host-ssl.yml and I'm unsure if for example in "ensure" when we you do:

if [[ -n "${SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL-}" ]] && [[ $SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL = "SSL" ]]

You don't prevent me from even passing in the described variables:

• SCHEMA_REGISTRY_KAFKASTORE_SASL_MECHANISM (set to SASL_PLAINTEXT)
• SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL (set to PLAINTEXT)
• SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS (set to a list of SASL_PLAINTEXT://...)

But even commenting that out and making sure I set ZOOKEEPER_SASL_ENABLED to false I'm unable to have Schema Registry connect.

This is the output until it finally gives up: https://gist.github.com/vascop/2a31b6b01000e5a5d3e181c3ae3348a4

So ideally, there would be a test for this scenario (SASL/PLAIN, no SSL) and/or some documentation about how to set it up. Can anyone help me figure out what might be missing?

[mishmam3]

Correct me if I'm wrong, but taking a look at the Kafka Consumers and producers used, it appears that they are getting their configurations from this file here: https://github.com/confluentinc/schema-registry/blob/master/core/src/main/java/io/confluent/kafka/schemaregistry/rest/SchemaRegistryConfig.java Looks like SASL/PLAIN is not supported and only SASL with Kerberos is supported for now.

By the way, did you end up using the schema registry for your use case?

[vascop]

@mishmam3 It's been almost 2 years but see my answer to my own question here: https://stackoverflow.com/q/45035991/505196

After a lot of testing (after we got all of it working) we ended up not using schema registry. My memory is a bit fuzzy on details, but there were enough quirky things that we didn't want to stick with it.

[chohoo89]

Please check the settings in Kafka. If you look at the Kafka log, it is most likely that your SASL/PLAIN settings are wrong. I also decided that it was not possible, but the Python client was not connected, so I touched this and that, and it was connected, and the Schema Registry was also connected.

I configured it with Docker. Hope this helps.

kafka1:
    image: confluentinc/cp-kafka:7.0.1
    container_name: kafka1
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      TZ: "Asia/Seoul"
      KAFKA_BROKER_ID: 1
      KAFKA_LISTENERS: SASL_PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka-1.kr:9092
      KAFKA_ZOOKEEPER_CONNECT: *kafkaZookeepers
      ZOOKEEPER_SASL_ENABLED: "false"
      KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf"
      KAFKA_INTER_BROKER_LISTENER_NAME: SASL_PLAINTEXT
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: 'LogAppendTime'
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: kafka-1.kr
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /srv/kafka_server_jaas.conf:/etc/kafka/kafka_server_jaas.conf
      - /srv/kafka1/data:/var/lib/kafka/data
      - /srv/kafka1/secrets:/etc/kafka/secrets
    networks: 
      - zookeeper-network
    restart: always

  schema-registry:
    image: confluentinc/cp-schema-registry:7.0.1
    container_name: schema-registry
    depends_on:
      - kafka1
    ports:
      - "8081:8081"
    environment:
      TZ: "Asia/Seoul"
      SCHEMA_REGISTRY_HOST_NAME: kafka-1.kr
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: *kafkaBrokers
      SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL: SASL_PLAINTEXT
      SCHEMA_REGISTRY_KAFKASTORE_SASL_MECHANISM: PLAIN
      SCHEMA_REGISTRY_KAFKASTORE_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="" password="";'
      SCHEMA_REGISTRY_LOG4J_LOGGERS: "org.apache.kafka=ERROR,io.confluent.rest.exceptions=FATAL"
      SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /srv/schema-registry/secrets:/etc/schema-registry/secrets
    networks: 
      - zookeeper-network