Support Youtube/Google one-touch authentication with Android

hanxue commented 7 years ago

When using 2FA to login to Youtube, there is either HTTP Error 500: Internal Server Error or HTTP Error 402.

I suspect it is because Google accounts with an active Android device allow the Android device to authorize sign in request, and 2FA is not required. This is what appears on my Android device:

google_signin

Could youtube-dl support this one-click sign on for Android when available, and fall back to TOTP/2FA only when it is not available?

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is

[x] I've verified and I assure that I'm running youtube-dl 2017.07.23

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):


$ youtube-dl -v 'https://www.youtube.com/watch?v=TAgVVc5hAiE'
[debug] System config: []
[debug] User config: ['--proxy', 'localhost:8888', '-uxxxxxxxx@gmail.com', '--mark-watched']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=TAgVVc5hAiE']
Type account password and press [Return]: 
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.07.23
[debug] Python version 3.6.1 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12, avprobe 12, ffmpeg 3.3.2, ffprobe 3.3.2, rtmpdump 2.4
[debug] Proxy map: {'http': 'localhost:8888', 'https': 'localhost:8888'}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]: 
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
WARNING: Unable to extract CheckCookie URL
[youtube] TAgVVc5hAiE: Downloading webpage

ghost commented 7 years ago

2FA support seems to be broken altogether, it doesn't work whether I use one-touch authorization or enter the TOTP code... The only way it works right now is by using --cookies.

Also, you may want to make the image smaller. Replace it with this:

[![Screenshot](https://i.imgur.com/eFc3eUhl.png)](https://i.imgur.com/eFc3eUh.png)

blimmo commented 6 years ago

Is 2FA still broken? You can stop google sending the prompt to your phone by disconnecting it from the internet. It will fall back to the normal authenticator codes but they don't seem to be working either.

hanxue commented 6 years ago

No, 2FA is still not working with the latest version of youtube-dl as of time of this response, 2018.05.18

$ proxychains4 youtube-dl --ignore-config -2 --mark-watched -u xxxxxxx@gmail.com -v 'https://www.youtube.com/watch?v=IYfgvS0FA7U'
[proxychains] config file found: /Users/hanxue/.proxychains/proxychains.conf
[proxychains] preloading /usr/local/Cellar/proxychains-ng/4.12_1/lib/libproxychains4.dylib
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--ignore-config', u'-2', u'--mark-watched', u'-u', u'PRIVATE', u'-v', u'https://www.youtube.com/watch?v=IYfgvS0FA7U']
Type account password and press [Return]: 
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.18
[debug] Python version 2.7.15 (CPython) - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 3.4.2, ffprobe 3.4.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
[youtube] IYfgvS0FA7U: Downloading webpage
[youtube] IYfgvS0FA7U: Downloading video info webpage
[youtube] IYfgvS0FA7U: Extracting video information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://r3---sn-npoeen7r.googlevideo.com/videoplayback?ipbits=0&fvip=3&dur=1172.704&expire=1526813590&gcr=sg&id=o-AI0GyqxD9yVgFGl3RaEPQ5EeqxqNOEJ1KLfruvFmMHF1&pl=63&mm=31%2C26&mn=sn-npoeen7r%2Csn-i3belne6&ip=2400%3A6180%3A0%3Ad0%3A%3Ae73%3A6001&requiressl=yes&ms=au%2Conr&mt=1526791855&mv=m&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgcr%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&keepalive=yes&ei=Nv8AW6eOIN2cowP9iYaIDw&c=WEB&lmt=1526290576056697&source=youtube&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&initcwndbps=180000&clen=284912645&itag=137&gir=yes&mime=video%2Fmp4&key=yt6&signature=C482C5FC9A4886E5F1A082EB1B5731C8D9CB4111.AA653F50927E2F151F7A7B217B8F74D303C69509&ratebypass=yes'

Note the HTTP error

WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request

Download still works though, just that I think authentication did not go through, and --mark-watched does not work. I check my YouTube history and that video does not appear.

hanxue commented 6 years ago

I have uploaded the dump files for YouTube / Google sign in at https://gist.github.com/hanxue/f5b62e6e940ab0e03a3cc60e5f130dd5

blimmo commented 6 years ago

Using a browser, going to https://accounts.google.com/ServiceLogin, which is the url used by YoutubeBaseInfoExtractor, redirects me to https://accounts.google.com/signin/v2/identifier?flowName=GlifWebSignIn&flowEntry=ServiceLogin.

Perhaps youtube-dl is using v1 signin which doesn't support 2FA anymore? I don't know how to test this as I can't access the signin page that youtube-dl seems to use through a browser.

Another thing: this problem seems to have moved away from the original subject of this issue. I think there are a few issues open that are related but perhaps we should make a new one purely about 2FA not working?

hanxue commented 6 years ago

The Android one-touch authentication is one of the few forms of 2FA supported by Google. If we are talking about 2FA, it could be OTP authenticator code, or code via phone/SMS. I am happy to expand the scope of this issue to cover all Google 2FA, not just Android one-touch authentication

blimmo commented 6 years ago

From the source: https://github.com/rg3/youtube-dl/blob/504f20dd302189db2cfe1cb5ee9a622c39ee693c/youtube_dl/extractor/youtube.py#L202

It seems that the authenticator app codes are supposed to be supported while other forms of 2FA (such as the one-touch authentication) aren't yet.

This is why I'm suggesting splitting the issue into one for the bug preventing the existing 2FA support working and one for the request for one-touch support

ytdl-org / youtube-dl