strimzi / kafka-kubernetes-config-provider

Kubernetes Configuration Provider for Apache Kafka®
Apache License 2.0
25 stars 12 forks source link

Secret is not properly read in #32

Open meretri opened 1 month ago

meretri commented 1 month ago

I am using Kafka Connect and have added the config provider like this:

config.providers: file,secrets
cconfig.providers.file.class: org.apache.kafka.common.config.provider.FileConfigProvider
config.providers.secrets.class: io.strimzi.kafka.KubernetesSecretConfigProvider

When I use the Secret Provider in my connector (Debezium), it works for some cases but not in others. Here it works:


schema.history.internal.producer.security.protocol: SSL
schema.history.internal.producer.ssl.truststore.location: /opt/kafka/external-configuration/kafka_truststore/kafka.truststore.jks
schema.history.internal.producer.ssl.truststore.password: ${secrets:kafka-connect/kafka-truststore:password}

But here it doesn't:

key.converter.apicurio.registry.request.ssl.truststore.location: /opt/kafka/external-configuration/schema_registry_truststore/ssl.truststore.jks
key.converter.apicurio.registry.request.ssl.truststore.password: ${secrets:kafka-connect/schema-registry-truststore:password}
key.converter.apicurio.registry.request.ssl.truststore.type: JKS

the error I get is:

Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: java.io.IOException: getSecretKey failed: Password is not ASCII

I think this is because it uses ${secrets:kafka-connect/schema-registry-truststore:password} as string and does not get the secret itself.

Does anyone have any idea why it works in the first case but not in the second?

scholzj commented 1 month ago

That sounds like you probably use it incorrectly? But without full logs and configurations, it is hard to be any more specific.

meretri commented 1 month ago

what kind of logs are you looking for? and which config are you missing? from the kafka connect?

scholzj commented 1 month ago

I guess the logs and configs of where you have this issue? It is not really clear how your setup looks like. If you use Stirmzi, then likely:

meretri commented 1 month ago

This is my setup

Here are the configs: 020-kafka-connect.txt 030-debezium-oracle-connector.txt

and the logs: logs_kafka_connect.txt logs_operator.txt

If you need anything else, I happy to provide more info

scholzj commented 1 month ago

That is neither full logs nor the custom resources as they are in the Kubernetes cluster.

You should maybe also check with Apicurio if they support configuration through config providers and how to debug what configuration they get?

meretri commented 1 month ago

so it could be, that the issue lies with Apicurio?

scholzj commented 1 month ago

I don't know. But that gives you the error. So I think that is the best point to start debugging it to see what it is gettting for data.

meretri commented 1 month ago

can you give me any pointers on how to debug this?

scholzj commented 1 month ago

No, I gave you the ideas I had above.