Proper handling of socket timeouts

[RubenKelevra]

Currently, socket timeouts are not properly handled (exception while handling an exception).

Uncaught exception from FUSE operation readdir, returning errno.EINVAL.
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 264, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=5001): Read timed out. (read timeout=120)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/http.py", line 249, in _do_request
    return self._session.request(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/requests_wrapper.py", line 223, in request
    return super(Session, self).request(method, url, *args, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=5001): Read timed out. (read timeout=120)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/fuse.py", line 734, in _wrapper
    return func(*args, **kwargs) or 0
  File "/usr/lib/python3.8/site-packages/fuse.py", line 960, in readdir
    for item in self.operations('readdir', self._decode_optional_path(path),
  File "/usr/lib/python3.8/site-packages/fuse.py", line 1076, in __call__
    return getattr(self, op)(*args)
  File "/usr/lib/python3.8/site-packages/ipfs_api_mount/__init__.py", line 163, in readdir
    return ['.', '..'] + list(self._ls(self.root + path).keys())
  File "/usr/lib/python3.8/site-packages/ipfs_api_mount/__init__.py", line 56, in ls
    for entry in api.ls(object_id)['Objects'][0]['Links']
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/client/base.py", line 15, in wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/client/files.py", line 393, in ls
    return self._client.request('/ls', args, decoder='json', **kwargs)
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/http.py", line 50, in wrapper
    return func(self, *args, **merged)
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/http.py", line 369, in request
    ret = self._request(method, url, params, parser, stream,
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/http.py", line 286, in _request
    res = self._do_request(method, url, params=params, stream=stream,
  File "/usr/lib/python3.8/site-packages/ipfshttpclient/http.py", line 253, in _do_request
    six.raise_from(exceptions.TimeoutError(error), error)
  File "<string>", line 3, in raise_from
ipfshttpclient.exceptions.TimeoutError: ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=5001): Read timed out. (read timeout=120)

[SupraSummus]

what is the expected behaviour? returning EAGAIN / EWOULDBLOCK?

[SupraSummus]

@RubenKelevra what is your version/commit of ipfs-api-mount? I've added timeout handling in 8b0ef38d12dc687deb4faa6762364431d679ba95, which is not yet released to pypi.

[RubenKelevra]

@RubenKelevra what is your version/commit of ipfs-api-mount? I've added timeout handling in 8b0ef38, which is not yet released to pypi.

I've used the released version: e316ff277ecebe73138fd98950a4ea903677ba06

[RubenKelevra]

what is the expected behaviour? returning EAGAIN / EWOULDBLOCK?

I don't know. It just felt too short until this happened. :)

[SupraSummus]

Ok, I feel this is already fixed but I'm not 100% sure. I'll add some tests for simulated timeouts and see whats going on.

[SupraSummus]

as mentioned in https://github.com/SupraSummus/ipfs-api-mount/issues/33#issuecomment-658859500 returning EAGAIN is risky, because it may be hidden from upper layer code. Lower level will retry the request over and over again.

I don't like the idea of returning EIO, because timeout is not an input-output error.

Good candidate may be implementing configurable timeout and returning ETIMEDOUT.