search

danielqsj / kafka_exporter

Kafka exporter for Prometheus
Apache License 2.0
2.17k stars 610 forks source link

Using on 2-way SSL secured Kafka Cluster, error reading /etc/kafka/secrets/ca-file, certificate and key must be supplied as a pair" #27

Closed IRobL closed 6 years ago

IRobL commented 6 years ago

Hi, thanks for uploading. I've been using this repo to great effect so far, but am encountering issues when running against an authenticating Kafka Cluster.

Now that my cluster has ACLs and client certs manditory, I need to figure out how to get this exporter to make use of client certs. When configuring the tls.ca-file switch, I try pointing it to a ca-file that looks something like this:

-----BEGIN CERTIFICATE-----
MIIDkzCCAnugAwIBAgIJAJtmWRyaaaaaaaaaaaaaaaaaaaaaaaaaaa...
................
.................==
-----END CERTIFICATE-----

But I get an error on boot:

error reading /etc/kafka/secrets/ca-file, certificate and key must be supplied as a pair"

After looking into things, it looks like maybe I need to point to a file that consists of server.crt and server.key? I'll be giving that a try tomorrow but thought I'd leave bread crumbs here as that it's the file types I've grown accustomed to using after working with Kafka are those .jks files.

IRobL commented 6 years ago

Ok, great news, I got it all working. So assuming you have a broker's .jks key, you can follow these steps to get the exporter working against 2-way client authenticating clusters. 

/usr/bin/keytool -importkeystore \
  -srckeystore /app/kafka_broker_secrets/broker1.keystore.jks \
  -destkeystore /app/kafka_broker_secrets/new-store.p12 \
  -deststoretype PKCS12 && \
  /usr/bin/openssl pkcs12 -in /app/kafka_broker_secrets/new-store.p12 \
    -nodes -nocerts \
    -out /app/kafka_broker_secrets/key.pem && \
  /usr/bin/openssl pkcs12 -in /app/kafka_broker_secrets/new-store.p12 -nokeys \
    -out /app/kafka_broker_secrets/cert.pem

those commands will dump out key.pem and cert.pem from a boker's jks file, eg broker1.keystore.jks. Thereafter, I use this command to boot up the exporter:

docker run \
    --net=host \
    --volume /app/kafka_broker_secrets:/etc/kafka/secrets \
    danielqsj/kafka-exporter:v1.0.1 \
      --kafka.server=broker1:29093 \
      --web.listen-address=:9308 \
      --tls.enabled \
      --tls.ca-file=/etc/kafka/secrets/ca-cert \
      --tls.cert-file=/etc/kafka/secrets/cert.pem \
      --tls.key-file=/etc/kafka/secrets/key.pem

Note that ca-cert is the CA I used to sign all the broker's SSL keys. Closing as that since this is in the issues list, anyone having trouble can simple look this issue up and figure out a solution.