search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
129.67k stars 9.78k forks source link

A suggestion to move username@password in URI into --proxy-user --proxy-pass #9450

Open CeruleanSky opened 8 years ago

CeruleanSky commented 8 years ago 
youtube-dl --verbose --proxy "socks5://username%%40domainname.com:P%%40ssWord@184.75.208.10:1080" https://www.youtube.com/watch?v=6W5pq4bIzIw
[youtube] 6W5pq4bIzIw: Downloading webpage
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'--proxy', u'socks5://username%%40domainname.com:P%%40ssWord@184.75.208.10:1080', u'https://www.youtube.com/watch?v=6W5pq4bIzIw']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.10 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-79000-g66edd86, ffprobe N-79000-g66edd86, rtmpdump 2.4
[debug] Proxy map: {u'http': u'socks5://username%%40domainname.com:P%40ssWord@184.75.208.10:1080', u'https': u'socks5://username%%40domainname.com:P%40ssWord@184.75.208.10:1080'}
Traceback (most recent call last):
  File "__main__.py", line 19, in <module>
  File "youtube_dl\__init__.pyo", line 421, in main
  File "youtube_dl\__init__.pyo", line 411, in _real_main
  File "youtube_dl\YoutubeDL.pyo", line 1735, in download
  File "youtube_dl\YoutubeDL.pyo", line 676, in extract_info
  File "youtube_dl\extractor\common.pyo", line 341, in extract
  File "youtube_dl\extractor\youtube.pyo", line 1216, in _real_extract
  File "youtube_dl\extractor\common.pyo", line 501, in _download_webpage
  File "youtube_dl\extractor\common.pyo", line 408, in _download_webpage_handle
  File "youtube_dl\extractor\common.pyo", line 388, in _request_webpage
  File "youtube_dl\YoutubeDL.pyo", line 1945, in urlopen
  File "urllib2.pyo", line 431, in open
  File "urllib2.pyo", line 449, in _open
  File "urllib2.pyo", line 409, in _call_chain
  File "youtube_dl\utils.pyo", line 932, in https_open
  File "urllib2.pyo", line 1194, in do_open
  File "httplib.pyo", line 1053, in request
  File "httplib.pyo", line 1093, in _send_request
  File "httplib.pyo", line 1049, in endheaders
  File "httplib.pyo", line 893, in _send_output
  File "httplib.pyo", line 855, in send
  File "youtube_dl\utils.pyo", line 898, in connect
  File "youtube_dl\socks.pyo", line 268, in connect
  File "youtube_dl\socks.pyo", line 264, in _make_proxy
  File "youtube_dl\socks.pyo", line 219, in _setup_socks5
  File "youtube_dl\socks.pyo", line 212, in _socks5_auth
youtube_dl.socks.Socks5Error: [Errno 1] unknown error

Description of your issue, suggested solution and other information

http proxy authentication works fine with this URI scheme, https has always failed, maybe it can be fixed also or made into another issue ticket.

Socks5 support seems to fail when trying to authenticate. I've included the verbose output. Also I suggest due to many usernames being email addresses taking the username/password parameters into their own command line switches. Otherwise users have to know that they need to encode their characters and be familiar with URI specification to use a username and password. Also decoding URIs can come with issues especially in experimental implementations.

This was done in a windows command prompt and I saved the parameters to a batch file so sometimes the % need additional escaping(%%40 for @ in a batch file, %40 from the command line). when composing the URIs

The suggestion is that the username and password be moved out to their own parameters, such as --proxy-user --proxy-pass

This way http,https,and socks5 proxies with passwords can be sure to work even with lots of @ signs, etc, and users don't have to guess which characters are allowed and are not, less likely to have to deal with shell + URI escaping rules(though if % is still used, that is another issue), and having to deal with both.

yan12125 commented 8 years ago

Please try this branch: https://github.com/yan12125/youtube-dl/tree/unquote-socks-credentials. It should correctly restore encoded values before passing to SOCKS servers.

CeruleanSky commented 8 years ago

@yan12125 Your patch works well. The uri scheme now behaves how it should and I was able to authenticate, at least for socks5 and download the video. Thank you.

I'll still leave this open in case somebody wants to move out the username/password into their own command line parameters to make it easier for other end users.

yan12125 commented 8 years ago

Thanks for helping testing! The fix will be included in the next version.

To developers: --proxy-user and --proxy-pass is useful, but it seems not easy to implement without Python's undocumented API _parse_proxy().