SSL issue: No such configuration property, couldn't create kafka producer

[carineKanouo]

Hi, i'm setting up prometheus-kafka adapter on a vm and i our kafka broker is reachable via https. when a fill kafkaSslClientCertFile, kafkaSslClientKeyFile, kafkaSslCACertFile parameters in config.go file, i get error like below

./prometheus-kafka-adapter {"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"","time":"2020-08-05T11:54:21+02:00"} {"level":"info","msg":"creating kafka producer","time":"2020-08-05T11:54:21+02:00"} {"error":"No such configuration property: \"ssl.certificate.location\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-08-05T11:54:21+02:00"}

sometimes i have the following error just by launching ./prometheus-kafka-adapter again, without modify anything else. it's random.

{"error":"No such configuration property: \"ssl.key.password\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-08-05T11:43:46+02:00"}

Can i have help please ?

[palmerabollo]

You shouldn't modify the code in config.go. Please try using the following env vars, as described in the readme.

KAFKA_SSL_CLIENT_CERT_FILE: Kafka SSL client certificate file, defaults to ""
KAFKA_SSL_CLIENT_KEY_FILE: Kafka SSL client certificate key file, defaults to ""
KAFKA_SSL_CLIENT_KEY_PASS: Kafka SSL client certificate key password (optional), defaults to ""
KAFKA_SSL_CA_CERT_FILE: Kafka SSL broker CA certificate file, defaults to ""

Please, let us know:

• what version of prometheus-kafka-adapter you are using.
• if you have the same issue running the Docker container.

[carineKanouo]

Hi, i'm using now the last commit of prometheus-kafka-adapter We are not working on container but on vm I used environment variable without modifying config.go file and i have these error

{"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"","time":"2020-08-06T11:19:20+02:00"} INFO[0000] creating kafka producer source="main.go:29" {"error":"Invalid value \"ssl\" for configuration property \"security.protocol\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-08-06T11:19:20+02:00"}

[palmerabollo]

I think something is wrong with your env vars. Could you please share all the env vars you are using? For example, the "invalid serialization format, using json" means that it detects an unknown value (empty string) for SERIALIZATION_FORMAT.

[carineKanouo]

Hi, "invalid serialization format, using json" is just a warning. I'm OK with JSON as default value. Anyway, i fixed it. and i still have the following error.

./prometheus-kafka-adapter INFO[0000] creating kafka producer source="main.go:29" {"error":"No such configuration property: \"ssl.certificate.location\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-08-07T14:43:22+02:00"}

Here are all my env vars: CK8A811N@noefyaz8> echo $KAFKA_BROKER_LIST gw.prod.hadoop:443 CK8A811N@noefyaz8> echo $KAFKA_SSL_CLIENT_CERT_FILE /etc/pki/tls/certs/prometheus-kafka-adapter_llt.pem CK8A811N@noefyaz8> echo $KAFKA_SSL_CLIENT_KEY_FILE /etc/pki/tls/private/prometheus-kafka-adapter_llt.key CK8A811N@noefyaz8> echo $KAFKA_SSL_CA_CERT_FILE /etc/pki/ca-trust/source/anchors/ca-bundle.crt CK8A811N@noefyaz8> echo $SERIALIZATION_FORMAT json CK8A811N@noefyaz8> echo $GIN_MODE debug CK8A811N@noefyaz8> echo $PORT 8081

[palmerabollo]

Looks good to me. SSL support was introduced in https://github.com/Telefonica/prometheus-kafka-adapter/pull/27. @jarett-cyxtera sorry to bother you, do you see something wrong or do you know why we could be gettint the error ""No such configuration property: "ssl.certificate.location""? I don't know why we get the "No such configuration property", it might be something related to librdkafka.

[jarett-cyxtera]

Hi @palmerabollo - we haven't been using the SSL functionality in a while, but I will try to replicate and see what's going on.

[carineKanouo]

Hello, Please do you have news about this issue. I made tests again and i have this output:

./prometheus-kafka-adapter {"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"","time":"2020-09-15T14:10:40+02:00"} {"level":"info","msg":"creating kafka producer","time":"2020-09-15T14:10:40+02:00"} {"error":"No such configuration property: \"ssl.key.password\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-09-15T14:10:40+02:00"}

./prometheus-kafka-adapter {"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"","time":"2020-09-15T14:17:00+02:00"} {"level":"info","msg":"creating kafka producer","time":"2020-09-15T14:17:00+02:00"} {"error":"Invalid value \"ssl\" for configuration property \"security.protocol\"","level":"fatal","msg":"couldn't create kafka producer","time":"2020-09-15T14:17:00+02:00"}

[palmerabollo]

@carineKanouo No news, I'm not able to reproduce it. I guess it's something wrong in your setup.

I've tried to run a docker container: docker run --rm -it telefonica/prometheus-kafka-adapter:1.7.0 sh

And then run prometheus-kafka-adapter with a correct value for SERIALIZATION_FORMAT:

export SERIALIZATION_FORMAT=json
./prometheus-kafka-adapter
{"level":"info","msg":"creating kafka producer","time":"2020-09-17T14:10:02Z"}

As you can see, no "invalid serialization format, using json" is shown. However, if I use a wrong value, it's shown in the output:

export SERIALIZATION_FORMAT=wrongvalue
./prometheus-kafka-adapter
{"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"wrongvalue","time":"2020-09-17T14:12:23Z"}
{"level":"info","msg":"creating kafka producer","time":"2020-09-17T14:12:23Z"}

In your examples, you say you use json:

echo $SERIALIZATION_FORMAT
json

But in the output, the env var value is empty:

{"level":"warning","msg":"invalid serialization format, using json","serialization-format-value":"","time":"2020-09-15T14:17:00+02:00"}

The same applies to all the other properties (ssl.key.password, etc). It looks like the env vars are not properly passed to the docker container you run.

[palmerabollo]

I'm closing this. I'm not able to reproduce it. I think something is wrong with your setup or the way you pass the required environment variables. Feel free to reopen the issue if you think we can help you debug the issue somehow.