Kyokoning / lofter_crawler

crawler
45 stars 7 forks source link

TimeoutError #4

Closed fsouls closed 4 years ago

fsouls commented 4 years ago

Traceback (most recent call last): File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 677, in urlopen chunked=chunked, File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 976, in _validate_conn conn.connect() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connection.py", line 370, in connect sslcontext=context, File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\util\ssl.py", line 377, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 401, in wrap_socket _context=self, _session=session) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 808, in init self.do_handshake() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 683, in do_handshake self._sslobj.do_handshake() TimeoutError: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没 有反应,连接尝试失败。

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\adapters.py", line 449, in send timeout=timeout File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 725, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\util\retry.py", line 403, in increment raise six.reraise(type(error), error, _stacktrace) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\packages\six.py", line 734, in reraise raise value.with_traceback(tb) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 677, in urlopen chunked=chunked, File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 381, in _make_request self._validate_conn(conn) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connectionpool.py", line 976, in _validate_conn conn.connect() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\connection.py", line 370, in connect sslcontext=context, File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ urllib3\util\ssl.py", line 377, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 401, in wrap_socket _context=self, _session=session) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 808, in init self.do_handshake() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 1061, in do_handshake self._sslobj.do_handshake() File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\ssl.py", line 683, in do_handshake self._sslobj.do_handshake() urllib3.exceptions.ProtocolError: ('Connection aborted.', TimeoutError(10060, ' 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。', None, 1 0060, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "tool/crawler.py", line 33, in save_tag(cfg, str(tag)) File "E:\Program files\lofter_crawler-master\tool..\lib\user_pigeonhole.py", line 208, in save_tag _capture_blog(single_header, url, hot, cfg) File "E:\Program files\lofter_crawler-master\tool..\lib\user_pigeonhole.py", line 101, in _capture_blog html = requests.get(url, headers).text File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\api.py", line 76, in get return request('get', url, params=params, kwargs) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\api.py", line 61, in request return session.request(method=method, url=url, kwargs) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\sessions.py", line 530, in request resp = self.send(prep, send_kwargs) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\sessions.py", line 643, in send r = adapter.send(request, kwargs) File "C:\Users\Nolan\AppData\Local\Programs\Python\Python36\lib\site-packages\ requests\adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError(10060, '由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。', None, 10060, None))

您好,试了几个几千热度的tag,都在爬到1000多条的时候出现了这个问题

Kyokoning commented 4 years ago

您好,感谢您的反馈!我更新了代码,您可以再一次尝试并且反馈结果吗?

fsouls commented 4 years ago

您好,试了两个5000参与的tag,很顺畅没有问题,非常感谢。