Nekmo / dirhunt

Find web directories without bruteforce
MIT License
1.76k stars 236 forks source link

Error in resp.raw.read(MAX_RESPONSE_SIZE, decode_content=True) #59

Closed Nekmo closed 5 years ago

Nekmo commented 5 years ago
Traceback (most recent call last):
  File "/home/nekmo/.virtualenvs/dirhunt/lib/python3.6/site-packages/urllib3/response.py", line 302, in _error_catcher
    yield
  File "/home/nekmo/.virtualenvs/dirhunt/lib/python3.6/site-packages/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/usr/lib/python3.6/http/client.py", line 449, in read
    n = self.readinto(b)
  File "/usr/lib/python3.6/http/client.py", line 483, in readinto
    return self._readinto_chunked(b)
  File "/usr/lib/python3.6/http/client.py", line 578, in _readinto_chunked
    chunk_left = self._get_chunk_left()
  File "/usr/lib/python3.6/http/client.py", line 546, in _get_chunk_left
    chunk_left = self._read_next_chunk_size()
  File "/usr/lib/python3.6/http/client.py", line 506, in _read_next_chunk_size
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nekmo/Workspace/dirhunt/dirhunt/exceptions.py", line 43, in wrapped
    return func(*args, **kwargs)
  File "/home/nekmo/Workspace/dirhunt/dirhunt/crawler_url.py", line 65, in start
    text = resp.raw.read(MAX_RESPONSE_SIZE, decode_content=True)
  File "/home/nekmo/.virtualenvs/dirhunt/lib/python3.6/site-packages/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/nekmo/.virtualenvs/dirhunt/lib/python3.6/site-packages/urllib3/response.py", line 307, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='cdn.databases.today', port=80): Read timed out.