Unable to start SRS service with external elasticsearch: unable to find valid certification path to requested target

[dekke046]

Describe the bug Using an external (company managed) ElasticSearch cluster throws a certificte error, this is due to the private CA which is being used within the company. The backingservices SRS helm chart does not provide an option to use a custom cacerts file.

To Reproduce Simple try connect to an external ElasticSearch cluster with a self-signed certificate

Expected behavior Having an option to specify a custom cacerts file, for example logstash has an option to use your own cacert file in the config:

  elasticsearch {
    hosts => "https://xxxx:9200"
    api_key => "xxxx:xxxx"
    data_stream => true
    ssl => true
    cacert => "/path/to/http_ca.crt" 
  }

Chart version What version of the charts are you using? 3.17.0

Have you made any customizations? Yes, unfortunately the values.yaml does not work very well in combination with OpenShift and the security policies which are configured as per company guidelines:

          runAsNonRoot: true
          allowPrivilegeEscalation: false
          capabilities:
            drop:
              - ALL
          seccompProfile:
            type: RuntimeDefault

Server (if applicable, please complete the following information): OpenShift kubernetes cluster, v1.25.16+9946c63

Additional context

[dekke046]

As a workaround I have adjusted srsservice_deployment.yaml, with a custom volume mount based on a secret containing the cacerts with the added ca's:

          volumeMounts:
            - mountPath: /etc/ssl/certs/java/
              name: cacerts
              readOnly: true

But making (too many) changes to template files is not the way to go

[kishorv10]

@reddy-srinivas @ravitejamekapega Can you please take a look?

[kishorv10]

@pegasystems/searchservice FYINA

[bhowd1]

Hi @dekke046,

Apologies for the late response.

We do provide two ways to mount your custom certificates to SRS in-case of External deployment of Elasticsearch. Please find them below:

1. You can use the Makefile's external-es-secrets target to generate a secret containing your certificate which is later referenced in SRS deployment. You can provide the certificate name and password as mentioned here.
2. You can also create your own secret containing the certificate and provide the secret name using values.yaml mentioned here.

You can also reference this section in Search and Reporting Service doc to know about the configs we support.

Please let us know in-case you face any issues.

Thanks

[dekke046]

Hi @bhowd1 ,

Its not about certificates for authentication, its about CA certificate for the initial TLS handshake. We are using regular authentication (username & password).

The company is using a private CA(in company singned certificate), once SRS tries to connect to this elasticsearch service it fails due to SSL Handshake issues, since it cant validate the certificate with the default included CA Certs in the docker SRS image.

[maracle6]

@dekke046 @bhowd1 I've deployed SRS in a similar environment, and the configuration is a bit counterintuitive for this scenario. The issue is that if you have srsStorage.tls.enabled set to false and srsStorage.basicAuthentication.enabled set to true the pod will not have the srsStorage.certificateName mounted. But if you have tls set to true and basic to false, it will mount the certificateName and still use your esCredentials.

So the procedure is:

1. Add a truststore to the srs-certificates secret. For this example, let's say you add trust.jks to this secret.
2. Set srsStorage.tls.enabled to true
3. Set srsStorage.certificateName to trust.jks
4. Set srsStorage.basicAuthentication.enabled to false
5. Set srsStorage.esCredentials.username and srsStorage.esCredentials.password

Now it will use your jks to validate the server and your esCredentials to authenticate.

There's one more requirement -- the JKS file must not have a password (there is no way to pass in a password to SRS), and it's a bit tricky to produce a truststore without a password. We created a JKS normally, then used https://keystore-explorer.org to remove the password.

[dekke046]

@dekke046 @bhowd1 I've deployed SRS in a similar environment, and the configuration is a bit counterintuitive for this scenario. The issue is that if you have srsStorage.tls.enabled set to false and srsStorage.basicAuthentication.enabled set to true the pod will not have the srsStorage.certificateName mounted. But if you have tls set to true and basic to false, it will mount the certificateName and still use your esCredentials.

So the procedure is:

1. Add a truststore to the srs-certificates secret. For this example, let's say you add trust.jks to this secret.
2. Set srsStorage.tls.enabled to true
3. Set srsStorage.certificateName to trust.jks
4. Set srsStorage.basicAuthentication.enabled to false
5. Set srsStorage.esCredentials.username and srsStorage.esCredentials.password

Now it will use your jks to validate the server and your esCredentials to authenticate.

There's one more requirement -- the JKS file must not have a password (there is no way to pass in a password to SRS), and it's a bit tricky to produce a truststore without a password. We created a JKS normally, then used https://keystore-explorer.org to remove the password.

@maracle6, Thanks for your guidance here, this is very helpful, with this explanation we can close this issue.