ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.2k stars 10.02k forks source link

Can't download from youtube #3241

Closed chmdznr closed 10 years ago

chmdznr commented 10 years ago

This problem happened on my two servers and not happening on my other server. (all servers have different public IP address).

Verbose output: root@debian:~# ./youtube-dl -v http://www.youtube.com/watch?v=ix3qdQkLnyY [debug] System config: [] [debug] User config: [] [debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=ix3qdQkLnyY'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2014.07.11.3 [debug] Python version 2.7.3 - Linux-3.2.0-4-amd64-x86_64-with-debian-7.5 [debug] Proxy map: {} [youtube] Setting language [youtube] ix3qdQkLnyY: Downloading webpage [youtube] ix3qdQkLnyY: Downloading video info webpage [youtube] ix3qdQkLnyY: Extracting video information [youtube] ix3qdQkLnyY: Encrypted signatures detected. [youtube] encrypted signature length 82 (40.41), itag 22, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 43, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 18, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 5, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 36, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 17, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 137, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 248, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 136, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 247, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 135, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 244, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 134, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 243, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 133, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 242, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 160, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 140, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 83 (40.42), itag 171, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 137, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 248, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 136, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 247, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 135, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 244, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 134, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 243, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 133, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 242, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 160, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 140, html5 player en_US-vflXGBaUN [youtube] encrypted signature length 82 (40.41), itag 171, html5 player en_US-vflXGBaUN [debug] Invoking downloader on u'https://r4---sn-vgqs7nez.googlevideo.com/videoplayback?expire=1405134000&ratebypass=yes&ipbits=0&ms=au&gcr=us&ip=142.54.172.171&mv=m&key=yt5&itag=22&source=youtube&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&upn=in3Io5AQfyk&fexp=902408%2C914084%2C922520%2C924213%2C924217%2C924222%2C924621%2C929305%2C930008%2C930817%2C934024%2C934030%2C936927%2C948504&mt=1405111835&mws=yes&sver=3&requiressl=yes&id=o-APNTPQ4AadeRe_vjtlwbNHlwrqAkp9sgJ7yF1_F2csLI&signature=5292CBB2B10ED8E3840D04EB1D2BFB1544705B7D.2E721833F7E5055964D7E7426BA2CE3DB695E0C6' ERROR: unable to download video data: '' Traceback (most recent call last): File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1022, in process_info success = dl(filename, info_dict) File "./youtube-dl/youtube_dl/YoutubeDL.py", line 998, in dl return fd.download(name, info) File "./youtube-dl/youtube_dl/downloader/common.py", line 291, in download return self.real_download(filename, info_dict) File "./youtube-dl/youtube_dl/downloader/http.py", line 58, in real_download data = self.ydl.urlopen(request) File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1227, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python2.7/urllib2.py", line 401, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 419, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain result = func(*args) File "./youtube-dl/youtube_dl/utils.py", line 591, in https_open return self.do_open(HTTPSConnectionV3, req) File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open r = h.getresponse(buffering=True) File "/usr/lib/python2.7/httplib.py", line 1034, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 407, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 371, in _read_status raise BadStatusLine(line) BadStatusLine: ''

chmdznr commented 10 years ago

Hmm.. managed to solve it using some "stupid" method. Apparently, my 2 servers is having trouble connecting to several googlevideo.com hosts (got zero reply). So, I changed the URL generated from youtube-dl with googlevideo.com that works and download the URL via wget. As illustration: on my previous post, my server is "rejected" by r4---sn-vgqs7nez.googlevideo.com. So, I changed that host into r4---sn-vgqs7n7k.googlevideo.com which is works fine for my servers. Weird case indeed.

phihag commented 10 years ago

Sorry, I don't follow. These two server names (r4---sn-vgqs7nez.googlevideo.com) are identical. In any case, this is probably not Google meddling with you (they usually send nice HTTP error messages), but some intermediate proxy.

jspiros commented 10 years ago

Those two server names aren't identical; one is "r4---sn-vgqs7nez", the other is "r4---sn-vgqs7n7k".

I was watching this bug because I'm running into a similar problem. All video downloads that I'm attempting that would touch "r6---sn-vgqs7nez" are failing. Though, I'm not having as much luck in replacing "sn-vgqs7nez" with "sn-vgqs7n7k"; I'm getting a 403 when I attempt that.

[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8                                                                                                                                       [290/7295]
[debug] youtube-dl version 2014.07.11.3
[debug] Python version 2.7.6 - Linux-3.2.0-4-amd64-x86_64-with-debian-jessie-sid
[debug] Proxy map: {}
[download] Downloading playlist: thebenheckshow
[youtube:user] thebenheckshow: Downloading video ids from 1 to 51
[youtube:user] thebenheckshow: Downloading video ids from 51 to 101
[youtube:user] thebenheckshow: Downloading video ids from 101 to 151
[youtube:user] playlist thebenheckshow: Downloading 144 videos
[download] Downloading video #1 of 144
[youtube] Setting language
[youtube] 6GcwcyL9LgM: Downloading webpage
[youtube] 6GcwcyL9LgM: Downloading video info webpage
[youtube] 6GcwcyL9LgM: Extracting video information
[youtube] 6GcwcyL9LgM: Searching for annotations.
[info] Writing video description to: /YouTube/users/thebenheckshow/20140711.Ben's ZX Spectrum Mod Part 1.6GcwcyL9LgM.mp4.description
[info] Writing video annotations to: /YouTube/users/thebenheckshow/20140711.Ben's ZX Spectrum Mod Part 1.6GcwcyL9LgM.mp4.annotations.xml
[info] Writing video description metadata as JSON to: /YouTube/users/thebenheckshow/20140711.Ben's ZX Spectrum Mod Part 1.6GcwcyL9LgM.info.json
[youtube] 6GcwcyL9LgM: Downloading thumbnail ...
[youtube] 6GcwcyL9LgM: Writing thumbnail to: /YouTube/users/thebenheckshow/20140711.Ben's ZX Spectrum Mod Part 1.6GcwcyL9LgM.jpg
[debug] Invoking downloader on u'https://r6---sn-vgqs7nez.googlevideo.com/videoplayback?mt=1405188880&upn=7gsE1uF6CSI&id=o-AMFVHL7IQaw0jSU7NGDLkq9h3j0R-3wkG07OV1P_aD-Z&requiressl=yes&ms=au&ipbits=0&expire=1405
213200&source=youtube&signature=3B5D6009DF36F311EB650D0DB250C0A062CF61A7.364D9FC3F94DD6C1F0AEE628ED02E0B39741699C&mv=m&sver=3&ratebypass=yes&sparams=id%2Cip%2Cipbits%2Citag%2Cratebypass%2Crequiressl%2Csource%2
Cupn%2Cexpire&ip=50.43.149.135&key=yt5&mws=yes&itag=22&fexp=902408%2C917000%2C924213%2C924217%2C924222%2C930008%2C934024%2C934030%2C936927'
ERROR: unable to download video data: ''
Traceback (most recent call last):
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1022, in process_info
    success = dl(filename, info_dict)
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 998, in dl
    return fd.download(name, info)
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 291, in download
    return self.real_download(filename, info_dict)
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/downloader/http.py", line 58, in real_download
    data = self.ydl.urlopen(request)
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/home/jspiros/.virtualenvs/video-dl/local/lib/python2.7/site-packages/youtube_dl/utils.py", line 591, in https_open
    return self.do_open(HTTPSConnectionV3, req)
  File "/usr/lib/python2.7/urllib2.py", line 1187, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1045, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 409, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 373, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''

I am assuming this is a problem on Google's end, and not youtube_dl's fault, as I'm also unable to load any of these videos in a web browser anywhere on my network, and looking at the source of the YouTube HTML pages my browsers are getting, they're also referencing the "r6---sn-vgqs7nez" server.

If @chmdznr was able to succeed in downloading from a different googlevideo.com server, I'm wondering if, in situations like this, there may be a way for youtube-dl to try and find another server, and attempt from that, before giving up?

chmdznr commented 10 years ago

Well, r4---sn-vgqs7nez.googlevideo.com is different from r4---sn-vgqs7n7k.googlevideo.com. Where did I get r4---sn-vgqs7n7k.googlevideo.com? It's from verbose output of youtube-dl that successfully downloading other youtube video. @jspiros you might try another Youtube video URL to find which googlevideo.com that works for you.

root@debian:~# ./youtube-dl --verbose "http://www.youtube.com/watch?v=HBxt_v0WF6Y"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.youtube.com/watch?v=HBxt_v0WF6Y']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.07.11.3
[debug] Python version 2.7.3 - Linux-3.2.0-4-amd64-x86_64-with-debian-7.5
[debug] Proxy map: {}
[youtube] Setting language
[youtube] HBxt_v0WF6Y: Downloading webpage
[youtube] HBxt_v0WF6Y: Downloading video info webpage
[youtube] HBxt_v0WF6Y: Extracting video information
[youtube] HBxt_v0WF6Y: Encrypted signatures detected.
[youtube] encrypted signature length 83 (42.40), itag 22, html5 player en_US-vflM1arS5
[youtube] HBxt_v0WF6Y: Downloading js player vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 43, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 18, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 5, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 36, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 17, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 137, html5 player en_US-vflM1arS5
[youtube] HBxt_v0WF6Y: Downloading js player vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 248, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 136, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 247, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 135, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 244, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 245, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 246, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 134, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 243, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 133, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 242, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 160, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 140, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 84 (41.42), itag 171, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 137, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 248, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 136, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 247, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 135, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 244, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 245, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 246, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 134, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 243, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 133, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 242, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 160, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 140, html5 player en_US-vflM1arS5
[youtube] encrypted signature length 83 (42.40), itag 171, html5 player en_US-vflM1arS5
[debug] Invoking downloader on u'https://r5---sn-vgqs7n7y.googlevideo.com/videoplayback?upn=kgZwctCt7kc&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&expire=1405224000&id=o-ALLlFTeUmPzgFcbwN7Ij3_463Uk9m1GrdQM5inQwMSqx&key=yt5&ip=142.54.172.171&ratebypass=yes&source=youtube&gcr=us&ms=au&itag=22&mws=yes&ipbits=0&requiressl=yes&sver=3&mv=m&mt=1405200371&fexp=902408%2C923347%2C924213%2C924217%2C924222%2C930008%2C934024%2C934030%2C935669%2C936927%2C943113%2C945000&signature=ED2528993CDCF9F772210AC066A7167068CDAE4B.25E30A4D49DDC015C912987C90E27F51F3D3AADC'
[download] Destination: Rihanna - Where Have You Been-HBxt_v0WF6Y.mp4
[download] 100% of 70.96MiB in 00:01

That one got another host that works on my server: r5---sn-vgqs7n7y.googlevideo.com.

jspiros commented 10 years ago

This appears to be a problem that many people are encountering with videos that are being fed from *vgqs7nez.googlevideo.com servers. See: