SSL verification failing

[vitekzach]

• PySeq version: 0.3.15
• Python version: 3.7.3
• Operating System: Windows 10 Enterprise

Description

I am trying to log events using seq on my python application. The logging fails on certificate verification.

I do have the certificate, because GET requests work as expected (e.g. I can browse logs in seq in browser).

What I Did

When I run seq locally, it works. When I try to reach seq deployed on a server, it fails on certificate verification.

import seqlog
import logging

seqlog.log_to_seq(
   server_url="https://******.com:5341/",
   level=logging.INFO,
   batch_size=1,
   auto_flush_timeout=1,  # seconds
   override_root_logger=True
)

seqlog.set_global_log_properties(
    application="testing_app",
)

logging.info("Hello, {name}!", name="World")

Here is the trace of the error I get:

--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\contrib\pyopenssl.py", line 456, in wrap_socket
    cnx.do_handshake()
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\OpenSSL\SSL.py", line 1915, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\OpenSSL\SSL.py", line 1647, in _raise_ssl_error
    _raise_current_error()
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\OpenSSL\_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\connection.py", line 344, in connect
    ssl_context=context)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\util\ssl_.py", line 347, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\contrib\pyopenssl.py", line 462, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\urllib3\util\retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='*****.com', port=5341): Max retries exceeded with url: /api/events/raw (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\seqlog\structured_logging.py", line 374, in publish_log_batch
    stream=True  # prevent '362'
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\requests\sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='*****', port=5341): Max retries exceeded with url: /api/events/raw (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
Call stack:
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\threading.py", line 1166, in run
    self.function(*self.args, **self.kwargs)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\seqlog\consumer.py", line 67, in flush
    self.callback(current_batch)
  File "C:\Users\vzach\miniconda3\envs\ml-general\lib\site-packages\seqlog\structured_logging.py", line 379, in publish_log_batch
    self.handleError(batch[0])
Message: 'Hello, {name}!'
Arguments: ()
Response from Seq was unavailable.

Is there a way of turning off SSL verification?

[tintoy]

Hi - this error vaguely rings a bell but I’ll need to look back through some old issues to find out why. I’ll take a look sometime this morning and get back to you 🙂

[tintoy]

RE: Disabling verification - the short answer is to set verify=False on the underlying Session object used by seqlog. This isn't explicitly supported at present (the Session is not exposed) but should be pretty easy to implement.

[tintoy]

BTW, I think Python on Windows doesn't necessarily use the same CA certificate store for SSL verification as your browser would.

It's been a while since I worked with this stuff but I think this link is still accurate:

https://stackoverflow.com/a/52961564

(not so much to override the CA certificate list file's path but see where that list file is in the first place)

[vitekzach]

Hi, thanks for the suggestion!

I am not a big fan of turning off the verification, especially since it involved editing the package.

I found another solution though - requests uses package called certifi that has a list of certificates, so all I had to do it export the relevant certificate (in my case root) in base-64 encoding and appending it to the cacert.pem file that is inside the certifi package.