tomasbedrich / pycaching

A Python 3 interface for working with Geocaching.com website.
https://pycaching.readthedocs.io/
GNU Lesser General Public License v3.0
61 stars 46 forks source link

`http.client.RemoteDisconnected` when running for a longer time #205

Closed BelKed closed 1 year ago

BelKed commented 1 year ago

The following error occurs every 12 hours at 8:15 and 20:15 when running the code for a longer time. (The code isn't that relevant, as it seems like the Geocaching website drops the connection at those times.)


I guess for solving this issue we could add some retry functionality to a Geocaching._request() function. The error occurs just once, so if we try to resend the request, everything should work fine...


Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 287, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pycaching/geocaching.py", line 82, in _request
    res = self._session.request(method, url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/discord/ext/tasks/__init__.py", line 101, in _loop
    await self.coro(*args, **kwargs)
  File "/Users/user/Documents/test.py", line 119, in gc
    for message in await process_caches():
  File "/Users/user/Documents/test.py", line 40, in process_caches
    new_caches_wps = tuple(get_caches())
  File "/usr/local/lib/python3.10/site-packages/pycaching/geocaching.py", line 440, in search_rect
    resp = self._request(self._urls["api_search"], params=params, expect="json")
  File "/usr/local/lib/python3.10/site-packages/pycaching/geocaching.py", line 94, in _request
    if e.response.status_code == 429:  # Handle rate limiting errors
AttributeError: 'NoneType' object has no attribute 'status_code'
BelKed commented 1 year ago

I'll submit a pull request in a moment if I fix the problem...

FriedrichFroebel commented 1 year ago

This sounds like some basic session timeout and nothing which should affect regular users (as the session usually will not be kept open for such long times in my opinion).

Edit: This would not explain why this works the second time, unless the session is being recreated in-between.

BelKed commented 1 year ago

This sounds like some basic session timeout and nothing which should affect regular users (as the session usually will not be kept open for such long times in my opinion).

I agree :D