Connection to Kafka with SASL for Sentry containers

[gwyn-bl]

Self-Hosted Version

24.6.0

CPU Architecture

x86_64

Docker Version

20.10.16

Docker Compose Version

2.6.0

Steps to Reproduce

Hi! I'm setting self-hosted Sentry 24.6.0 and struggling to config it with Kafka with authentication.

Here is configs: sentry/sentry.conf.py

DEFAULT_KAFKA_OPTIONS = {
        "bootstrap.servers": externalKafka.host.io:9093,
        "message.max.bytes": 50000000,
        "socket.timeout.ms": 1000,
        "sasl.mechanism": "SCRAM-SHA-512",
        "security.protocol": "SASL_SSL",
        "sasl.username": " ",
        "sasl.password": " ",
    }

relay/config.yml

kafka_config:
        - name: "bootstrap.servers"
          value: "externalKafka.host.io:9093"
        - name: "message.max.bytes"
          value: 50000000
        - name: "sasl.mechanism"
          value: "SCRAM-SHA-512"
        - name: "security.protocol"
          value: "SASL_SSL"
        - name: "sasl.username"
          value: " "
        - name: "sasl.password"
          value: " "

Expected Result

Sentry component (sentry-subscription-consumer, sentry-ingest-consumer, sentry-worker, etc.) connect to Kafka-cluster using provided config.

Actual Result

Snuba and Relay has no problem with connection, but Sentry services sentry-subscription-consumer, sentry-ingest-consumer, sentry-worker giving me this error: 6|1723028162.828|FAIL|rdkafka#consumer-1| [thrd:externalKafka.host.io:9093/bootstrap]: externalKafka.host.io:9093/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 0ms in state APIVERSION_QUERY, 4 identical error(s) suppressed)

Sentry containers cannot connect to kafka with auth, despite I provided Sentry creds and security protocol. Looks like Sentry components doesn't able to use them. So I'm wondering, how to provide config correctly or Sentry just doesn't support external Kafka with authentication

Here is the merge which allows SSL for Relay Allow enabling SSL for Kafka. (https://github.com/getsentry/relay/pull/3232)

So I assumed that Sentry itself should support it but can't find any examples or docs how to make it work.

Please, tell me what am I doing wrong?

Event ID

No response

[aldy505]

By any chance are you using self-signed (or custom) CA certificate installed on Kafka? If that's the case, you might want to define the path to CA file as specified on the Python's Kafka library https://github.com/confluentinc/confluent-kafka-python?tab=readme-ov-file#ssl-certificates Other than that, your config looks fine.

If that doesn't solve your problem, can you specify what Kafka version are you running on?

I don't configure an external Kafka on my Sentry, and I only have experience using SASL_PLAINTEXT on Redpanda though, but I believe the principles are the same.

[gwyn-bl]

Thanks for the answer! I gonna try to specify CA and if it doesn't help try to switch to plaintext auth in kafka, see if it would work.. Please, don't close issue for now

[Alena86]

Try to restructure the DEFAULT_KAFKA_OPTIONS to be:

DEFAULT_KAFKA_OPTIONS = {
    "common": {
            "bootstrap.servers": externalKafka.host.io:9093,
            "message.max.bytes": 50000000,
            "socket.timeout.ms": 1000,
            "sasl.mechanism": "SCRAM-SHA-512",
            "security.protocol": "SASL_SSL",
            "sasl.username": " ",
            "sasl.password": " ",
    },
    "producer": {any values  that  you want to set specifically for the producer},
    "consumer": {any values that  you want to set specifically for the consumer}
}

This did the trick for me.