kafka receiver sasl + tls auth config

[povilasv]

Component(s)

receiver/kafka

What happened?

Description

If you want to use sasl with tls atm it's rather unclear how to set it up.

In order to enable tls atm we have a check for:

    if config.TLS != nil {
        if err := configureTLS(*config.TLS, saramaConfig); err != nil {
            return err
        }
    }

So if you have kafka with sasl and tls you expect this:

    auth:
      sasl:
        username: "user"
        password: "secret"
        mechanism: "SCRAM-SHA-512"

to work, but it doesn't.

You also need to set something in tls structure, so it gets enabled. In this case I set insecure to it's default value and then connection works:

    auth:
      sasl:
        username: "user"
        password: "secret"
        mechanism: "SCRAM-SHA-512"
    tls:
      insecure: false

This is also noted in sarama sasl example - https://github.com/IBM/sarama/blob/main/examples/sasl_scram_client/main.go#L92-L108C1

Where they both set tls.enabled=true and sasl.enabled=true

I suggest maybe we can add:

tls:
  enabled: false

as default? And then users wanting to configure tls, can set it to enabled?

Thougths?

Collector version

v0.95.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions[bot]]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[am-oxipay]

After doing a bit of debug it looks like if you want to use SCRAM + PLAIN_TEXT auth without tls you need to omit the TLS stanza .

This works for me but it's taken me a lot of time and effort to work out.

receivers:
  kafka:
    resolve_canonical_bootstrap_servers_only: false
    auth:
      sasl:
        username:  <username>
        password:  <password>
        mechanism: SCRAM-SHA-256
        version: 1
      #tls:
        #insecure: true
        #insecure_skip_verify: true        
    brokers:
      - "myserver.com:9092"
    encoding: text
    client_id: uniqueid
    initial_offset: latest
    autocommit:
      enable: false
      interval: 30s
    topic: foo

[povilasv]

Yeah.. We really need some better config UX here. ATM basically if you want SCRAM + PLAIN TEXT, you need to get rid of tls config:

    auth:
      sasl:
        username: "user"
        password: "secret"
        mechanism: "SCRAM-SHA-512"
    #tls:
    #  insecure: false

if you want SCRAM + TLS, you need to set something in tls:

    auth:
      sasl:
        username: "user"
        password: "secret"
        mechanism: "SCRAM-SHA-512"
    tls:
      insecure: false

@pavolloffay / @MovieStoreGuy would appreciate ant feedback / direction what should we do here :)

[guanlinz]

Thanks, AWS MSK could use this config to establish with otel as well

[am-oxipay]

I suspect that if you are using TLS or want to use TLS it would work.

From my investigation, it's only in the scenario where you aren't using TLS (like me where I am just testing and ironically didn't want the complexity) then it needs to be absent.