Failure to connect to the confluent schema registry

[kanissh]

Describe the issue I am facing the error PKIX path building failed while connecting to confluent schema registry when running kpow-ce in Docker.

I was wondering is there a property in kpow I could set so as to skip the SSL check when connecting to the schema registry. Or any other permanent solution to resolve this issue is also appreciated.

[d-t-w]

Hi @kanissh thanks for your question.

There are two ways of resolving this issue, you can:

1. Enable SSL by configuring a truststore correctly; or,
2. Skip certificate validation.

Configure SSL For Kpow and Confluent Schema Registry

In your case you will want to configure a truststore containing the root CA SSL certificate.

Kpow CE supports the standard SSL connection parameters for Confluent Schema Registry, their documentation is here:

https://docs.confluent.io/platform/current/schema-registry/security/index.html#additional-configurations-for-https

We provide access to those parameters as environment variables, you can see the full list of Confluent Schema Registry environment variables supported by Kpow here:

https://docs.factorhouse.io/kpow-ee/config/schema-registry/#confluent-schema-registry

The relevant ones in your case are listed under 'Confluent SSL Configuration'.

The only catch is you must provide the truststore to the docker container in a way that it is accessible to the Kpow process running within the container. E.g. you have to add the truststore to the container directory system.

There are lots of ways of doing that in a production system like using secrets, sidecar containers, volume mounts etc.

The easiest way in a development/test setup is to simply define a volume mount when you run Kpow with docker, I use a command like this:

docker run --volume="$PWD/dev-resources:/dev-resources" -p 3000:3000 -m 2G --env-file ./docker/confluent.env factorhouse/kpow-ce:92.2

The --volume flag mounts a disk volume from $PWD/dev-resources on my host machine to /dev-resources inside the docker container, I have the truststore in my host machine directory and then I can reference it inside my confluent.env configuration file like:

SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION=/dev-resources/my-truststore.jks

Or you can turn off certificate validation entirely:

Skip Certificate Validation for Confluent Schema Registry

Note: this approach definitely works when connecting to a Kafka Cluster, I expect it works with Schema Registry too as they are both implemented the same way and have exactly the same configuration parameters.

Note 2: this approach leaves you open to man-in-the-middle attacks as you are deliberately not confirming the authenticity of the certificate provided by the schema registry endpoint, you are just using the certificate to perform encryption.

In your confluent.env or equivalent docker config file, have the entry:

SCHEMA_REGISTRY_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=

That is the key for the ssl-endpoint-identification-algorithm and an empty value. That effectively turns off cert validation.

Hopefully that helps! Derek

PS. Just a quick note that I realised while resolving this ticket that the full set of supported Schema Registry configuration is not clearly defined in our CE documentation, rather it all sits under our EE product configs. CE supports this full set as well so I will improve the documentation next week.

[kanissh]

@d-t-w thank you for pointing out the options.

Using SCHEMA_REGISTRY_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM in the environment variable file did not resolve the issue. It still fails with the same error.

But the property ssl.endpoint.identification.algorithm: '' is added as an input by kpow, visible on the console log.

[kanissh]

@d-t-w

Unfortunately adding keystore and truststore also did not work for me. I followed the following tutorial to create the truststore and keystore - https://docs.confluent.io/platform/current/security/security_tutorial.html#security-tutorial

I used the following script to create them as highlighted in the above doc. Link to the script in the doc is as follows - https://github.com/confluentinc/confluent-platform-security-tools/blob/master/kafka-generate-ssl.sh

[d-t-w]

Hi @kanissh, connecting Kpow to SSL enabled Confluent Schema registry is a fairly common deployment.

Can you connect your regular Kafka clients to your SSL enabled Schema Registry? If so Kpow will also connect as long as your configuration is the same and the binary keystore files are available within the docker container and configured correctly.

When I connect to our local SSL enabled Confluent Schema Registry I use the following config:

ENVIRONMENT_NAME=Trade Book (Staging)
BOOTSTRAP=kafka-1:19092,kafka-2:19093,kafka-3:19094

SCHEMA_REGISTRY_NAME=Local Schema
SCHEMA_REGISTRY_URL=https://schema-1:8081
SCHEMA_REGISTRY_AUTH=USER_INFO
SCHEMA_REGISTRY_USER=admin
SCHEMA_REGISTRY_PASSWORD=admin
SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION=/docker/ssl/my-truststore.jks
SCHEMA_REGISTRY_SSL_TRUSTSTORE_PASSWORD=password

LICENSE_ID=<your-licence-details>
...
...
...

I run the Kpow CE docker container with a command similar to:

docker run --volume="$PWD/docker:/docker" --network compose_default -p 3000:3000 -m 2G --env-file ./docker/local-schema-ssl.env factorhouse/kpow-ce:92.2.1

That works fine, with Kpow CE connecting to the SSL enabled confluent schema registry.

If you have a specific Kpow issue please send over the application logs.