Cannot connect to ElasticSearch running on HTTPS

[revmannl]

Trying the plugin from Hassio on Pi but cannot connect to an Elasticsearch running on HTTPS. I got the right certificates configured on ES (Let's Encrypt) connecting with browser is no issue. All efforts result in SSL errors. Config seems to ignore the verify_ssl = false option.

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 485, in wrap_socket
    cnx.do_handshake()
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1934, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1671, in _raise_ssl_error
    _raise_current_error()
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 394, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 491, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 172, in perform_request
    response = self.pool.urlopen(method, url, body, retries=Retry(False), headers=request_headers, **kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 376, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 394, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 491, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 170, in _async_setup_component
    hass, processed_config
  File "/config/custom_components/elastic/__init__.py", line 93, in async_setup
    publisher = DocumentPublisher(conf, gateway, hass, system_info)
  File "/config/custom_components/elastic/__init__.py", line 268, in __init__
    self._create_index_template()
  File "/config/custom_components/elastic/__init__.py", line 456, in _create_index_template
    es_version = self._gateway.get_es_version()
  File "/config/custom_components/elastic/__init__.py", line 160, in get_es_version
    version = self.client.info()["version"]
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 241, in info
    return self.transport.perform_request('GET', '/', params=params)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 178, in perform_request
    raise SSLError('N/A', str(e), e)
elasticsearch.exceptions.SSLError: ConnectionError(("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)) caused by: SSLError(("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",))

Is this part on my end, or something with the elastic add-on in HA?

[jakommo]

From the error it looks like the cert is not trusted, which should not be the case with verify_ssl = false. Are there any other messages logged? i.e. something about the verify_ssl = false not getting picket up or so?

[revmannl]

By looking at the logging from HA, there are no messages that the verify_ssl = falseis not getting picked up. Running the 'curl' command in the homeassistant docker also states that it cannot verify the legitimacy of the server. Running 'curl' with the--insecure option works. Somehow the docker doesn't seem to have the right certs in its truststore.

Are there other places where I could find any errors regarding the python script or HA not picking up the verify_ssl = false ?

[revmannl]

jakommo... after changing the config of elastic to use different cert and changing the HA configuration to url = '<servername:port>' with the quotes and verify_ssl = true , it is working!

After reading some more about the LetsEncrypt cert it appears I was using the wrong cert :/ Changing that to the right one, 'curl' in the homeassistant docker started working, but needed to change the url = '<servername:port>' to use those quotes, without them, same error that cert was no good. Doesn't seem to make sence though, but I am keeping this config as I am getting data into Elastic. Thanks for the reply, and thanks for the Integration to Elastic!!

[legrego]

Thanks @jakommo for helping out. Glad you got this sorted, and thanks for posting what finally worked for you @revmannl!

I'm going to close this since it appears you have everything up and running now 👍