search

seglo / kafka-lag-exporter

Monitor Kafka Consumer Group Latency with Kafka Lag Exporter
Apache License 2.0
650 stars 191 forks source link

There is no clear documentation on how to connect to a ssl and kerberos enabled Kafka cluster with the helm chart #321

Open gshilo opened 2 years ago

gshilo commented 2 years ago

Describe the bug There is no clear documentation on how to connect to a ssl and kerberos enabled Kafka cluster with the helm chart.

To Reproduce I am sorry, but I cannot elaborate the actual values.yaml file I used. Please see the additional context for details.

Environment

Additional context Hello

I am trying to install kafka-lag-exporter in an OpenShift cluster using the helm chart. The kafka cluster I want to monitor is Cloudera based and uses SASL_SSL (with kerberos) but lag-exporter cannot connect to it. Lag exporter's readme file does not contain clear instructions for setting up sasl or ssl via the helm values file. So I tried searching the issues section and found some references like https://github.com/seglo/kafka-lag-exporter/issues/270 and https://github.com/seglo/kafka-lag-exporter/issues/263

In those issues I can see JSON configuration files that contains ssl parameters. It looks like they refer to the standalone lag exporter. However, using the helm chart I expect to configure everything only in the values.yaml file. I tried to copy the parameters from those JSON files. First I added my keystore, truststore and keytab files to the secrets section of the values file. Then I configured the clusters section like this: clusters:

and the same params for adminClientProperties.

But when I try to run lag exporter it complains that it does not recognise those parameters and I can see messages like this in the log: The configuration 'ssl.truststore.location' was supplied but isn't a known config. This is thrown for each and every ssl or sasl parameter I pass.

So what is the righ way to configure lag exporter to work with ssl and kerberos using the helm chart ?

panthdesai18 commented 2 years ago

Hey, did this get to work by any means? I am also stuck having the same issue with SASL mechanism as SCRAM-SHA-512

seglo commented 2 years ago

I'm sorry, but I lack the environment to set this up and experience to verify its correctness. I recently added minikube integration tests. If you can setup a reproducer integration test by configuring strimzi and your best attempt at kafka lag exporter config I can look into it in more detail.

sdahlbac commented 2 years ago

It is not exactly the same thing but I struggled quite some time with SASL_SSL, sasl plain with a cert-manager created certificate, but finally got it working:

clusters:
  - adminClientProperties:
      sasl.jaas.config: >-
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="..." password="...";
      sasl.mechanism: PLAIN
      security.protocol: SASL_SSL
      ssl.truststore.location: /opt/docker/secrets/truststore.jks
    adminClientPropertiesNoQuotes:
      ssl.truststore.password: ...
    bootstrapBrokers: >-
      kafka-0.kafka-headless.core.svc.cluster.local:9094,kafka-1.kafka-headless.core.svc.cluster.local:9094,kafka-2.kafka-headless.core.svc.cluster.local:9094
    consumerProperties:
      sasl.jaas.config: >-
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="..." password="...";
      ssl.truststore.location: /opt/docker/secrets/truststore.jks
    consumerPropertiesNoQuotes:
      ssl.truststore.password: ...
    name: local-cluster
extraMounts:
  - mount:
      secret:
        secretName: client-tls-secret
    mountPath: /opt/docker/secrets/
    name: client-tls-secret

or as terraform

resource "helm_release" "kafka-lag-exporter" {
  name             = "kafka-lag-exporter"
  namespace        = "core"
  create_namespace = true

  repository = "https://seglo.github.io/kafka-lag-exporter/repo/"
  chart      = "kafka-lag-exporter"

  set {
    name  = "pollIntervalSeconds"
    value = "10"
  }

  set {
    name  = "clusters[0].name"
    value = "local-cluster"
  }

  # bootstrap brokers isn't really sensitive, but tfe_outputs marks everything sensitive
  set_sensitive {
    name  = "clusters[0].bootstrapBrokers"
    value = replace(data.tfe_outputs.base.values.kafka-url, ",", "\\,")
  }

  set {
    name  = "clusters[0].adminClientProperties.sasl\\.mechanism"
    value = "PLAIN"
  }
  set {
    name  = "clusters[0].adminClientProperties.security\\.protocol"
    value = "SASL_SSL"
  }
  set {
    name  = "clusters[0].consumerProperties.ssl\\.truststore\\.location"
    value = "/opt/docker/secrets/truststore.jks"
  }
  set_sensitive {
    name  = "clusters[0].consumerPropertiesNoQuotes.ssl\\.truststore\\.password"
    value = ...
  }
  set {
    name  = "clusters[0].adminClientProperties.ssl\\.truststore\\.location"
    value = "/opt/docker/secrets/truststore.jks"
  }
  set_sensitive {
    name  = "clusters[0].adminClientPropertiesNoQuotes.ssl\\.truststore\\.password"
    value = ...
  }
  set_sensitive {
    name  = "clusters[0].consumerProperties.sasl\\.jaas\\.config"
    value = "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"...\" password=\"...\";"
  }
  set_sensitive {
    name  = "clusters[0].adminClientProperties.sasl\\.jaas\\.config"
    value = "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"...\" password=\"...\";"
  }
  set {
    name  = "extraMounts[0].name"
    value = "client-tls-secret"
  }
  set {
    name  = "extraMounts[0].mountPath"
    value = "/opt/docker/secrets/"
  }
  set {
    name  = "extraMounts[0].mount.secret.secretName"
    value = "client-tls-secret"
  }
}

hope this helps