Cerfiticate Verification failed with any kind of request to any endpoint

[nocturn9x]

I am getting an ssl error while trying to connect to https://api.telegram.org. The weird thing is that when I tried with a minimal example to https://www.google.com/ I got a different error, which got me confused. Below some more details

First Try - Telegram API

This code would throw ssl.SSLCertVerificationError even with a custom ssl_context (e.g. ssl.create_default_context()) or with verify=False (which is indeed weird, but I couldn't figure out why this parameter is ignored by reading the library source code)

data = {"offset": 0, "timeout": 15, "limit": 100, "allowed_updates": []}
response = await asks.post(f"https://api.telegram.org/bot{self.token}/getUpdates", params=data)

Full Traceback

[DEBUG - 09/03/2020 18:07:21 PM] Starting authorization
Traceback (most recent call last):
  File "/Users/mattia/Desktop/PyTG/test.py", line 8, in <module>
    bot.start()
  File "/Users/mattia/Desktop/PyTG/PyTG/bot.py", line 212, in start
    trio.run(self._authorize_bot)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/trio/_core/_run.py", line 1804, in run
    raise runner.main_task_outcome.error
  File "/Users/mattia/Desktop/PyTG/PyTG/bot.py", line 178, in _authorize_bot
    response = await asks.post(f"https://api.telegram.org/bot{self.token}/getUpdates", params=data, verify=False)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/base_funcs.py", line 30, in request
    r = await s.request(method, url=uri, **kwargs)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 168, in request
    connection_timeout, self._grab_connection, url)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/utils.py", line 15, in timeout_manager
    return await coro(*args)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 368, in _grab_connection
    sock = await self._make_connection(host_loc)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 342, in _make_connection
    sock, port = await self._connect(host_loc)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 101, in _connect
    (host, int(port))), port
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 79, in _open_connection_https
    tls_standard_compatible=False)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/__init__.py", line 404, in connect_tcp
    await stream.start_tls()
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/_networking.py", line 316, in start_tls
    not self._tls_standard_compatible)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/_networking.py", line 171, in start_tls
    self._raw_socket.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)

Second Try - A minimal (failing) example

Considering that I couldn't figure out why this was happening I tried with a minimal example, which just made me more confused.

import trio
async def main():
    req = await asks.get("https://www.google.com")
    return req

trio.run(main)

which, again, throws ssl.SSLCertVerificationError, but with a different message and slightly different traceback, as follows

Traceback

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/trio/_core/_run.py", line 1804, in run
    raise runner.main_task_outcome.error
  File "<stdin>", line 2, in main
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/base_funcs.py", line 30, in request
    r = await s.request(method, url=uri, **kwargs)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 168, in request
    connection_timeout, self._grab_connection, url)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/utils.py", line 15, in timeout_manager
    return await coro(*args)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 368, in _grab_connection
    sock = await self._make_connection(host_loc)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 342, in _make_connection
    sock, port = await self._connect(host_loc)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 101, in _connect
    (host, int(port))), port
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/asks/sessions.py", line 79, in _open_connection_https
    tls_standard_compatible=False)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/__init__.py", line 404, in connect_tcp
    await stream.start_tls()
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/_networking.py", line 316, in start_tls
    not self._tls_standard_compatible)
  File "/Users/mattia/Desktop/envs/PyTG/lib/python3.7/site-packages/anyio/_networking.py", line 171, in start_tls
    self._raw_socket.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)

My environment specs

Python Version (virtual environment): 3.7.4 Library Version: 2.3.6

Other, probably, useful information

OS: Mac OS X Yosemite 10.10.5 My pip freeze output:

anyio==1.2.3
asks==2.3.6
async-generator==1.10
attrs==19.3.0
certifi==2019.11.28
chardet==3.0.4
h11==0.9.0
idna==2.9
outcome==1.0.1
requests==2.23.0
sniffio==1.1.0
sortedcontainers==2.1.0
trio==0.13.0
urllib3==1.24.1

P.S.: I opened a new issue to get more visibility

[theelous3]

wut

I'll investigate.

[theelous3]

>>> import asks
>>> import trio
>>> 
>>> async def wut():
...     r = await asks.get("https://www.google.com")
...     print(r.status_code)
...
>>> trio.run(wut)
200

and

(env) ~/.../python/asks >>> pip freeze              ±[A2●][master]
anyio==1.2.3
-e git+https://github.com/theelous3/asks.git@918a0a9d56ad528e1a7036fd6a644473c2b8420d#egg=asks
async-generator==1.10
attrs==19.3.0
curio==1.1
h11==0.9.0
idna==2.9
outcome==1.0.1
sniffio==1.1.0
sortedcontainers==2.1.0
trio==0.13.0

(Using the same asks' code as upstream bar some unused imports)

[theelous3]

I am currently baffled.

Have you got any kind of custom configs locally for https / certs?

[carlbordum]

Try making a new release?

https://github.com/theelous3/asks/compare/2.3.6...master

[theelous3]

lol, yeah

Done.

[nocturn9x]

I am currently baffled.

Have you got any kind of custom configs locally for https / certs?

No, I dont, even though I was tipped to install the certifi library, the problem now seems fixed, I'll test in my environment soon

[theelous3]

It's possible there was an unseen bit of debug code or something in the last release. Going to take a proper look and make sure it doesn't happen again.

Please close the issue if it's resolved for you :)

[nocturn9x]

Also it's worth noting that the verify parameters gets ignored

Il giorno mar 17 mar 2020 alle ore 16:14 M.J. notifications@github.com ha scritto:

It's possible there was an unseen bit of debug code or something in the last release. Going to take a proper look and make sure it doesn't happen again.

Please close the issue if it's resolved for you :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/theelous3/asks/issues/158#issuecomment-600126508, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFS6TPLW2ZOBVLYM6UMFSH3RH6HXFANCNFSM4LEZW7DA .

[daisylb]

I'm seeing the same problem on asks 2.4.7:

❯ cat test.py
import trio
import asks

async def test():
    await asks.get('https://example.com')

trio.run(test)

❯ poetry run python test.py
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    trio.run(test)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/trio/_core/_run.py", line 1896, in run
    raise runner.main_task_outcome.error
  File "test.py", line 5, in test
    await asks.get('https://example.com')
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/base_funcs.py", line 30, in request
    r = await s.request(method, url=uri, **kwargs)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 204, in request
    connection_timeout, self._grab_connection, url
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/utils.py", line 15, in timeout_manager
    return await coro(*args)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 405, in _grab_connection
    sock = await self._make_connection(host_loc)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 379, in _make_connection
    sock, port = await self._connect(host_loc)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 105, in _connect
    return await self._open_connection_https((host, int(port))), port
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 82, in _open_connection_https
    tls_standard_compatible=False,
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/__init__.py", line 407, in connect_tcp
    await stream.start_tls()
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/_networking.py", line 324, in start_tls
    not self._tls_standard_compatible)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/_networking.py", line 171, in start_tls
    self._raw_socket.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1076)

❯ poetry show | egrep "asks|trio|certifi"
asks             2.4.7     asks - async http
certifi          2020.6.20 Python package for providing Mozilla's CA Bundle.
trio             0.16.0    A friendly Python library for async concurrency and I/O

[daisylb]

Also, if I change the URL to https://google.com the error message changes to unable to get local issuer certificate, same as above:

❯ poetry run python test.py
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    trio.run(test)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/trio/_core/_run.py", line 1896, in run
    raise runner.main_task_outcome.error
  File "test.py", line 5, in test
    await asks.get('https://google.com')
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/base_funcs.py", line 30, in request
    r = await s.request(method, url=uri, **kwargs)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 204, in request
    connection_timeout, self._grab_connection, url
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/utils.py", line 15, in timeout_manager
    return await coro(*args)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 405, in _grab_connection
    sock = await self._make_connection(host_loc)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 379, in _make_connection
    sock, port = await self._connect(host_loc)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 105, in _connect
    return await self._open_connection_https((host, int(port))), port
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/asks/sessions.py", line 82, in _open_connection_https
    tls_standard_compatible=False,
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/__init__.py", line 407, in connect_tcp
    await stream.start_tls()
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/_networking.py", line 324, in start_tls
    not self._tls_standard_compatible)
  File "/Users/leigh/Library/Caches/pypoetry/virtualenvs/leighbrenecki-tu8Qjx8h-py3.7/lib/python3.7/site-packages/anyio/_networking.py", line 171, in start_tls
    self._raw_socket.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)

[theelous3]

I'm not seeing this issue at all. Tests OK and manual check off https with goodle OK

>>> import asks
>>> import curio
>>> async def main():
...     r = await asks.get("https://google.com")
...     print(r.status_code)
...
>>> curio.run(main)
200
>>>

Maybe uninstall your dependancies and reinstall. I pushed a new minor ver in the meantime, v.2.4.8, though it shouldn't change anything.

[daisylb]

Yeah, trying in a fresh environment works fine. That's... weird 🤷‍♀️

[theelous3]

Likely just some updatey stuff in anyio or another dep :)

I gave anyio a rough pin in the last setup.py change, to avoid this a little more in the future (I believe a v2 change will happen soon).