Request connection failed. Exiting...

Plurmo commented 2 years ago

- Using Windows 10

Issue

I had an issue when trying to run the code. After I entered the URL into the console, an error message appeared: "Request connection failed. Exiting..."

Output

C:\Users\User\AppData\Roaming\Python\Python39\site-packages\grequests.py:22: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util.ssl_ (C:\\Users\\User\\AppData\\Roaming\\Python\\Python39\\site-packages\\urllib3\\util\\ssl_.py)', 'urllib3.util (C:\\Users\\User\\AppData\\Roaming\\Python\\Python39\\site-packages\\urllib3\\util\\__init__.py)']. 
  curious_george.patch_all(thread=False, select=False)
[>] Enter the coursehero url: https://www.coursehero.com/file/39711060/BIO-101-Class-Notes-2docx/
[>] Request connection failed. Exiting...

I then removed the try-except statement from the code @ Line 88 and ran it again. This was the error I received: "RecursionError: maximum recursion depth exceeded"

Output:

C:\Users\User\AppData\Roaming\Python\Python39\site-packages\grequests.py:22: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (C:\\Users\\User\\AppData\\Roaming\\Python\\Python39\\site-packages\\urllib3\\util\\__init__.py)', 'urllib3.util.ssl_ (C:\\Users\\User\\AppData\\Roaming\\Python\\Python39\\site-packages\\urllib3\\util\\ssl_.py)']. 
  curious_george.patch_all(thread=False, select=False)
[>] Enter the coursehero url: https://www.coursehero.com/file/39711060/BIO-101-Class-Notes-2docx/
Traceback (most recent call last):
  File "c:\Users\User\Downloads\CourseHeroUnblur.py", line 89, in <module>
    resp = session.get(url, headers=make_headers())
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 555, in get    
    return self.request('GET', url, **kwargs)
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\sessions.py", line 655, in send   
    r = adapter.send(request, **kwargs)
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\connection.py", line 397, in connect
    self.ssl_context = create_urllib3_context(
  File "C:\Users\User\AppData\Roaming\Python\Python39\site-packages\urllib3\util\ssl_.py", line 312, in create_urllib3_context
    context.options |= options
  File "C:\Program Files\Python39\lib\ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "C:\Program Files\Python39\lib\ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "C:\Program Files\Python39\lib\ssl.py", line 602, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 492 more times]
RecursionError: maximum recursion depth exceeded

How-To Fix

I then went on StackOverflow here (first solution) and found out that I needed to add these two lines of code at the top of the code:

import gevent.monkey
gevent.monkey.patch_all()

After inserting those two lines at the top of the code, everything seems to work fine.

Conclusion

I don't know why the code isn't working before the fix. I thought I would just let you know here and perhaps help others who might have the same issue. I have to say, I love this program. It's working great and I appreciate your time and effort.

daijro commented 2 years ago

Hello, I'm curious about what would be causing the issue here, seeing that grequests actually does run gevent.monkey.patch_all upon being imported, (with theading and select patching off) before importing anything else that could trigger the ssl error.

Does requests-html work for you without having to run gevent.monkey patch? For example:

>>> from requests_html import HTMLSession
>>> session = HTMLSession()
>>> session.get('https://1.1.1.1')
<Response [200]>

Thank you for bringing my attention to this! I might have to update my other active projects if this is an issue.

Plurmo commented 2 years ago

I found out the problem. I have an extension in my ide that reorganizes the order of imports. This was causing the issue. Sorry for the trouble.

daijro / CourseHeroUnblur