search

elastic / connectors

Source code for all Elastic connectors, developed by the Search team at Elastic, and home of our Python connector development framework
https://www.elastic.co/guide/en/enterprise-search/master/index.html
Other
72 stars 126 forks source link

Allow insecure HTTPS #2491

Open dodie opened 5 months ago

dodie commented 5 months ago

Problem Description

When I try to connect to an Elasticsearch cluster with HTTPS, it is only possible to do so with a certificate with valid CA that is set up via the ca_certs configuration option. In my case, the target Elasticsearch is a test instance used for trials only, but it only accepts HTTPS traffic. For ad-hoc tests it would be convenient to access it without verifying the certificates. (Similarly to how curl can be used with the --insecure flag.)

Proposed Solution

Allow setting the verify_certs property via the connector config.yml, e.g.:

connectors:
-
  ...

elasticsearch:
  host: "https://<ES_HOST>:9200"
  verify_certs: false    # Defaults to true
  api_key: "..."

Additional Context

As workaround to allow insecure access via HTTPS, one might locally change the following flag to False:

https://github.com/elastic/connectors/blob/bed2e63cd2c01a15e33c13024b7a9aac5935d1be/connectors/es/client.py#L93

Goldini500 commented 4 months ago

We are currently facing the exact same issue. Unfortunately changing options["verify_certs"] to "False" and then rebuilding the container does not change anything.

We are getting the same issue:

[FMWK][15:44:34][INFO] Waiting for Elasticsearch at https://elasticsearch-test-cluster-es-http.mgt.svc:9200 (so far: 1 secs) [FMWK][15:44:34][ERROR] Could not connect to the Elasticsearch server [FMWK][15:44:34][ERROR] Cannot connect to host elasticsearch-test-cluster-es-http.mgt.svc:9200 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')]

@dodie is this working for you?