apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.45k stars 3.7k forks source link

Add docs for Kafka SASL / SCRAM configuration to Apache Kafka Ingestion #10572

Open techdocsmith opened 3 years ago

techdocsmith commented 3 years ago

If Kafka is configured with SASL / SCRAM, then Druid, which is the Kafka consumer, should pass the SASL / SCRAM credentials in the consumerProperties block of ioConfig section of the Kafka supervisor spec:

"ioConfig": {
      "topic": "some_topic",
      "consumerProperties": {
         "bootstrap.servers": "host1:9092,host2:9092,host3:9092",
         "sasl.mechanism": "SCRAM-SHA-256",
         "security.protocol": "SASL_SSL",
         "sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"some_userID\" password=\"some_password\";"
      }
      "taskCount": 1,
      "replicas": 1,
      "taskDuration": "PT1H"
}

cc: @druid-matt , @petermarshallio

Might be a good issue for a community contributor.

layoaster commented 2 years ago

It's not supported by now?

FrankChen021 commented 2 years ago

The issue here I think is to make some improvement on the doc. You can pass necessary properties required by the SASL in the consumerProperties as above

layoaster commented 2 years ago

indeed it seems it's related to the lack of documentation.