Twitch WinError 10060 and Timeouts

gyvess commented 7 years ago

[x] I've verified and I assure that I'm running youtube-dl 2016.10.16
[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
What is the purpose of your issue?
[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[x] Question
[ ] Other

WinError 10060

C:\WINDOWS\system32>youtube-dl.exe https://www.twitch.tv/toushinsaiofficialtvjp/v/95015481
[debug] System config: []
[debug] User config: ['-o', 'D:/Downloads/youtubedownload/videos/%(title)s[%(fps)sFPS].%(ext)s', '--ffmpeg-location', 'D:\\Do
wnloads\\youtubedownload\\avconv\\win32\\usr\\bin', '--cache-dir', 'D:/Downloads/youtubedownload/videos/cache', '--no-check-c
ertificate', '-f', 'bestvideo[fps=60][height<1080]+bestaudio[ext=m4a]/best', '--sub-langs', 'en-GB,en', '--no-mtime', '--no-p
art', '-i', '-R', 'infinite', '--fragment-retries', 'infinite', '-v']
[debug] Command-line args: ['https://www.twitch.tv/toushinsaiofficialtvjp/v/95015481']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2016.10.12
[debug] Python version 3.4.4 - Windows-XP-5.1.2600-SP3
[debug] exe versions: avconv 11.3, avprobe 11.3
[debug] Proxy map: {}
[twitch:vod] 95015481: Downloading vod info JSON
[twitch:vod] 95015481: Downloading vod access token
[twitch:vod] 95015481: Downloading m3u8 information
[debug] Invoking downloader on 'http://vod.edgecast.hls.ttvnw.net/v1/AUTH_system/vods_a44b/toushinsaiofficialtvjp_23413784896
_532032446/chunked/index-dvr.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 16404
[download] Destination: D:\Downloads\youtubedownload\videos\【闘神祭2016 Champions Carnival】 10月15日（土） 1日目[NAFPS].mp4
[download]  49.4% of ~11.88GiB at  5.51MiB/s ETA 16:56         ERROR: unable to download video data: <urlopen error [WinError
 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or establish
ed connection failed because connected host has failed to respond>
Traceback (most recent call last):
  File "C:\Python\Python34\lib\urllib\request.py", line 1183, in do_open
  File "C:\Python\Python34\lib\http\client.py", line 1137, in request
  File "C:\Python\Python34\lib\http\client.py", line 1182, in _send_request
  File "C:\Python\Python34\lib\http\client.py", line 1133, in endheaders
  File "C:\Python\Python34\lib\http\client.py", line 963, in _send_output
  File "C:\Python\Python34\lib\http\client.py", line 898, in send
  File "C:\Python\Python34\lib\http\client.py", line 871, in connect
  File "C:\Python\Python34\lib\socket.py", line 516, in create_connection
  File "C:\Python\Python34\lib\socket.py", line 507, in create_connection
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a perio
d of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\YoutubeDL.py", line 1699, in p
rocess_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\YoutubeDL.py", line 1641, in d
l
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\downloader\common.py", line 35
3, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\downloader\hls.py", line 115,
in real_download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\downloader\common.py", line 35
3, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\downloader\http.py", line 61,
in real_download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\YoutubeDL.py", line 2001, in u
rlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 482, in _open
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp0kv97by3\build\youtube_dl\utils.py", line 870, in http_o
pen
  File "C:\Python\Python34\lib\urllib\request.py", line 1185, in do_open
urllib.error.URLError: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not proper
ly respond after a period of time, or established connection failed because connected host has failed to respond>

Timeout

C:\WINDOWS\system32>Waiting for file handle to be closed ...
Updated youtube-dl to version 2016.10.16.
youtube-dl.exe https://www.twitch.tv/toushinsaiofficialtvjp/v/95015481
[debug] System config: []
[debug] User config: ['-o', 'D:/Downloads/youtubedownload/videos/%(title)s[%(fps)sFPS].%(ext)s', '--ffmpeg-location', 'D:\\Do
wnloads\\youtubedownload\\avconv\\win32\\usr\\bin', '--cache-dir', 'D:/Downloads/youtubedownload/videos/cache', '--no-check-c
ertificate', '-f', 'bestvideo[fps=60][height<1080]+bestaudio[ext=m4a]/best', '--sub-langs', 'en-GB,en', '--no-mtime', '--no-p
art', '-i', '-R', 'infinite', '--fragment-retries', 'infinite', '-v']
[debug] Command-line args: ['https://www.twitch.tv/toushinsaiofficialtvjp/v/95015481']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2016.10.16
[debug] Python version 3.4.4 - Windows-XP-5.1.2600-SP3
[debug] exe versions: avconv 11.3, avprobe 11.3
[debug] Proxy map: {}
[twitch:vod] 95015481: Downloading vod info JSON
[twitch:vod] 95015481: Downloading vod access token
[twitch:vod] 95015481: Downloading m3u8 information
[debug] Invoking downloader on 'http://vod.edgecast.hls.ttvnw.net/v1/AUTH_system/vods_a44b/toushinsaiofficialtvjp_23413784896
_532032446/chunked/index-dvr.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 16404
[download] Destination: D:\Downloads\youtubedownload\videos\【闘神祭2016 Champions Carnival】 10月15日（土） 1日目[NAFPS].mp4
[download]  50.8% of ~11.88GiB at  7.14MiB/s ETA 13:01         ERROR: unable to download video data: timed out
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\YoutubeDL.py", line 1699, in p
rocess_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\YoutubeDL.py", line 1641, in d
l
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\downloader\common.py", line 35
3, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\downloader\hls.py", line 115,
in real_download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\downloader\common.py", line 35
3, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpna76borx\build\youtube_dl\downloader\http.py", line 165,
 in real_download
  File "C:\Python\Python34\lib\http\client.py", line 537, in read
  File "C:\Python\Python34\lib\http\client.py", line 576, in readinto
  File "C:\Python\Python34\lib\socket.py", line 378, in readinto
socket.timeout: timed out

When downloading VODs from Twitch, I frequently run into WinErrors or Timeouts. The download will hang for quite some time and then print the problem. The trouble with this is that I have to monitor the download to ensure that it's still downloading, if it isn't repeat the command, and then wait a long time to check what percentage of the file I've already downloaded then begin getting the file again.

Is there an argument I can use that would allow youtube-dl to automatically start over if data is not received for a set amount of time, preferably remembering where it left off so it doesn't have to check what's already been partially downloaded. If not I'd like to make a feature request for it. That way I can just set it and forget it.

This has only happened to me on Twitch to be clear.

Cinerar commented 7 years ago

I'm expiriencing same issue on some urls on YouTube not sure but probably connected with #10761

HASJ commented 7 years ago

Same thing with very recently uploaded YouTube videos.

weedy commented 7 years ago

youtube-dl obviously has a retry mechanism.

[download]  79.2% of ~9.08GiB at  1.68MiB/s ETA 03:12[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 404: Not Found. Retrying fragment Frag6952 (attempt 10 of 10)...
[download] Skipping fragment Frag6952...

It just doesn't catch everything

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 453, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 409, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 480, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/lib/python2.7/ssl.py", line 734, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 621, in read
    v = self._sslobj.read(len or 1024)
SSLError: ('The read operation timed out',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/hls.py", line 115, in real_download
    success = ctx['dl'].download(frag_filename, {'url': frag_url})
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 354, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 61, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 2001, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/site-packages/youtube_dl/utils.py", line 1034, in https_open
    req, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
URLError: <urlopen error timed out>

rekcah1986 commented 6 years ago

PS E:> youtube-dl.exe https://www.youtube.com/watch?v=clU8c2fpk2s -v [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.youtube.com/watch?v=clU8c2fpk2s', u'-v'] [debug] Encodings: locale cp936, fs mbcs, out cp936, pref cp936 [debug] youtube-dl version 2018.06.19 [debug] Python version 2.7.14 (CPython) - Windows-10-10.0.17134 [debug] exe versions: ffmpeg 3.4 [debug] Proxy map: {} [youtube] clU8c2fpk2s: Downloading webpage ERROR: Unable to download webpage: <urlopen error [Errno 10060] > (caused by URLError(error(10060, ''),)) File "d:\tools\python27\lib\site-packages\youtube_dl\extractor\common.py", line 598, in _request_webpage return self._downloader.urlopen(url_or_request) File "d:\tools\python27\lib\site-packages\youtube_dl\YoutubeDL.py", line 2211, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "d:\tools\python27\lib\urllib2.py", line 429, in open response = self._open(req, data) File "d:\tools\python27\lib\urllib2.py", line 447, in _open '_open', req) File "d:\tools\python27\lib\urllib2.py", line 407, in _call_chain result = func(*args) File "d:\tools\python27\lib\site-packages\youtube_dl\utils.py", line 1101, in https_open req, **kwargs) File "d:\tools\python27\lib\urllib2.py", line 1198, in do_open raise URLError(err)

dirkf commented 3 months ago

Stale.

ytdl-org / youtube-dl

Twitch WinError 10060 and Timeouts #10935

What is the purpose of your issue?