dixudx / tumblr-crawler

Easily download all the photos/videos from tumblr blogs. 下载指定的 Tumblr 博客中的图片,视频
1.15k stars 345 forks source link

Max retries exceeded with url: /api/read?type=photo&num=50&start=0 #34

Open geosmart opened 7 years ago

geosmart commented 7 years ago

is tumblr has limit the max request?

Traceback (most recent call last):
  File "tumblr-photo-video-ripper.py", line 288, in <module>
    CrawlerScheduler(sites, proxies=proxies)
  File "tumblr-photo-video-ripper.py", line 149, in __init__
    self.scheduling()
  File "tumblr-photo-video-ripper.py", line 162, in scheduling
    self.download_media(site)
  File "tumblr-photo-video-ripper.py", line 165, in download_media
    self.download_photos(site)
  File "tumblr-photo-video-ripper.py", line 176, in download_photos
    self._download_media(site, "photo", START)
  File "tumblr-photo-video-ripper.py", line 193, in _download_media
    proxies=self.proxies)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 513, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 645, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 212, in resolve_redirects
    **adapter_kwargs
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 623, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='demo.tumblr.com', port=443): Max retries exceeded with url: /api/read?type=photo&num=50&start=0 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x6ffff8269d0>: Failed to establish a new connection: [Errno 116] Connection timed out',))
dixudx commented 7 years ago

Haven't met this before. You can retry it after a while.

geosmart commented 7 years ago

hello,I retried,but the same error. so I developed a new branch of tumblr-crawler with tumblr API V2,after some test case that make sure everything is ok, then I will commit to your git repository.

geosmart commented 7 years ago

i folked your project,and upload a new branch of tumblr-crawler-apiv2

HuangSecondary commented 5 years ago

It is probably caused by the socks or/and http port setting. You should check the port you used in your computer.

LiLejia commented 5 years ago

Try this

pip install ndg-httpsclient
pip install pyopenssl
pip install pyasn1

https://stackoverflow.com/questions/33410577/python-requests-exceptions-sslerror-eof-occurred-in-violation-of-protocol