ross / requests-futures

Asynchronous Python HTTP Requests for Humans using Futures
Other
2.11k stars 152 forks source link

On exception it returns xx (Caused By None), whereas the requests module returns xx (Caused By xx) #127

Closed iamsstef closed 1 year ago

iamsstef commented 1 year ago

When an exception is thrown it does return (Caused By None) so i can't know the details of the error. For example that's what requests module throws on that specific error: HTTPSConnectionPool(host='*********', port=443): Max retries exceeded with url: ******* (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

and here is what requests futures throws: None: Max retries exceeded with url: ***** (Caused by None)

ross commented 1 year ago

Can you provide a snippet of code you're using when you run into the error? I've tried and have been unable to reproduce that behavior, e.g. if I hit something that has a self-signed cert:

#!/usr/bin/env python3

from requests_futures.sessions import FuturesSession

session = FuturesSession()
future = session.get('https://self-signed.badssl.com/')
try:
    future.result()
except Exception as e:
    print(e)
$ ./test.py
HTTPSConnectionPool(host='self-signed.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:997)')))
ross commented 1 year ago

I get the same result with as_completed:


from concurrent.futures import as_completed
from requests_futures.sessions import FuturesSession

session = FuturesSession()
futures = [
    session.get('https://self-signed.badssl.com/'),
    session.get('https://untrusted-root.badssl.com/'),
]

for future in as_completed(futures):
    try:
        future.result()
    except Exception as e:
        print(e)
$ test.py
HTTPSConnectionPool(host='untrusted-root.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))
HTTPSConnectionPool(host='self-signed.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:997)')))

This is with python 3.10

$ python --version
Python 3.10.8
iamsstef commented 1 year ago

tbh i switched to aiohttp because i couldn't figure it out, i'm trying to reproduce it again now within my code, because out of my script's function seems to produce the expected error. If i remember correctly the function was being called via asyncio.create_task. I will close the issue and if i will manage to reproduce it i'll re-open it. Thank you for your time.

ross commented 1 year ago

Ah, problems generally arrise when mixing async types. It may be that asyncio and python threads/futures don't place nicely with each other.