tv.youtube.com doesn't work, or 2fa isn't working quite right, or both

coldwaterq commented 7 years ago

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2017.08.18. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2017.08.18

Before submitting an issue make sure you have:

[X] At least skimmed through the README, most notably the FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[X] Bug report (encountered problems with youtube-dl)
[X] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

python -m youtube_dl --verbose -u REDACTED@gmail.com -w --ffmpeg-location ffmpeg -f worst -o %(upload_date)s.%(uploader)s.%(title)s.%(ext)s "https://tv.youtube.com/?v=VQ4H33iIPWY"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-u', 'PRIVATE', '-w', '--ffmpeg-location', 'ffmpeg', '-f', 'worst', '-o', '%(upload_date)s.%(uploader)s.%(title)s.%(ext)s', 'https://tv.youtube.com/?v=VQ4H33iIPWY']
Type account password and press [Return]:
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2017.08.18
[debug] Python version 3.6.2 - Windows-10-10.0.15063-SP0
[debug] exe versions: ffmpeg N-76526-ga5202bc, ffprobe N-76526-ga5202bc
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
WARNING: Unable to extract CheckCookie URL
[youtube] VQ4H33iIPWY: Downloading webpage
[youtube] VQ4H33iIPWY: Downloading video info webpage
ERROR: VQ4H33iIPWY: YouTube said: This video is not available.
Traceback (most recent call last):
  File "C:\Users\ColdwaterQ\AppData\Local\Programs\Python\Python36-32\lib\site-packages\youtube_dl\YoutubeDL.py", line 776, in extract_info
    ie_result = ie.extract(url)
  File "C:\Users\ColdwaterQ\AppData\Local\Programs\Python\Python36-32\lib\site-packages\youtube_dl\extractor\common.py", line 433, in extract
    ie_result = self._real_extract(url)
  File "C:\Users\ColdwaterQ\AppData\Local\Programs\Python\Python36-32\lib\site-packages\youtube_dl\extractor\youtube.py", line 1537, in _real_extract
    expected=True, video_id=video_id)
youtube_dl.utils.ExtractorError: VQ4H33iIPWY: YouTube said: This video is not available.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://tv.youtube.com/?v=VQ4H33iIPWY
Show: https://tv.youtube.com/?browse=UCspnjcbufE7wey4CHTFrwmw

I don't think the show necessarily should be included because of the copy right issue, but for the videos I would like to be able to grab news clips now and then. This time I was hoping to get some of the footage of the eclipse

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information

I am not sure if this is a bug in 2fa, because google does the prompt on my phone, then I enter the code from authenticator, and in the verbose output I see 500 ERROR. So that could be it, or it could be trying to get the video on youtube.com instead of tv.youtube.com, which isn't the same even though the links use the same parameters.

If you need me to try anything else or look up anything just let me know and I will try to help.

dstftw commented 7 years ago

Post the output with --dump-pages.

yan12125 commented 7 years ago

Most likely a duplicate of #13759, https://github.com/rg3/youtube-dl/issues/1336#issuecomment-312462483 and #9857. Google Prompt is not supported yet.

coldwaterq commented 7 years ago

I created a new account without 2fa and I still get "youtube_dl.utils.ExtractorError: VQ4H33iIPWY: YouTube said: This video is not available." I can do --dump-pages, but I prefer to not submit my account information, is there a way to dump pages without disclosing anything personal?

coldwaterq commented 7 years ago

But looking at the --dump-pages output I get the line. "[youtube] Dumping request to https://www.youtube.com/watch?v=VQ4H33iIPWY&gl=US&hl=en&has_verified=1&bpctr=9999999999". And since TV content can't be loaded on the standard youtube domain I think that the problem is just that www.youtube.com/watch?v=VQ4H33iIPWY != tv.youtube.com/watch?v=VQ4H33iIPWY

coldwaterq commented 7 years ago

I hacked the youtube.py extractor on my computer to force it to use tv.youtube.com instead of www.youtube.com and it is getting the unauthenticated tv.youtube.com page instead of the video when it goes to https://tv.youtube.com/watch?v=VQ4H33iIPWY&gl=US&hl=en&has_verified=1&bpctr=9999999999 and https://tv.youtube.com/get_video_info. So I think something with the authentication may need to be modified along with detecting tv vs not tv youtube links

coldwaterq commented 7 years ago

Is there a way I can help with the geo-restriction since I am in the area it is restricted to?

yan12125 commented 7 years ago

The fastest way should be dumped files. There won't be passwords but your username and personal preferences may be leaked.

coldwaterq commented 7 years ago

This is the youtube.py that works for me on tv.youtube.com. https://gist.github.com/coldwaterq/0462b9411c72bea49883cbd8e9c1c72a

judge2020 commented 6 years ago

I can confirm the above youtube.py works for making a connection to tv.youtube.com (without 2fa), however, there are issues with the content you download (possibly related to DRM).

Content issue:

The first 10 seconds, where the USA logo appears, are fine, however, after that, it's likely DRM kicks in for the actual content and displays this

While Windows 10 movies & TV ( as well as windows media player) aims to decode the file at any costs, VLC freezes on the last frame and stays there while the counter continues.

Diff of the youtube.py files: https://www.diffchecker.com/TwxSZzvI

download output

[debug] System config: []
[debug] User config: [u'--netrc']
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://tv.youtube.com/?v=Bq6ug91Qeoc']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.10.20
[debug] Python version 2.7.14 - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-88042-g72c3d9ae45, ffprobe N-88042-g72c3d9ae45
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
[youtube] Checking cookie
[youtube] Bq6ug91Qeoc: Downloading webpage
[youtube] Bq6ug91Qeoc: Downloading video info webpage
[youtube] Bq6ug91Qeoc: Extracting video information
WARNING: unable to extract uploader nickname
[youtube] Bq6ug91Qeoc: Downloading embed webpage
[youtube] {146} signature length 40.41, html5 player vflN-B5oM
[youtube] {145} signature length 40.41, html5 player vflN-B5oM
[youtube] {224} signature length 40.41, html5 player vflN-B5oM
[youtube] {144} signature length 40.41, html5 player vflN-B5oM
[youtube] {222} signature length 40.41, html5 player vflN-B5oM
[youtube] {223} signature length 40.41, html5 player vflN-B5oM
[youtube] {143} signature length 40.41, html5 player vflN-B5oM
[youtube] {142} signature length 40.41, html5 player vflN-B5oM
[youtube] {161} signature length 40.41, html5 player vflN-B5oM
[youtube] {148} signature length 40.41, html5 player vflN-B5oM
[youtube] {149} signature length 40.41, html5 player vflN-B5oM
[youtube] {150} signature length 40.41, html5 player vflN-B5oM
[youtube] Bq6ug91Qeoc: Downloading MPD manifest
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://r5---sn-5uaeznde.googlevideo.com/videoplayback?gcr=us&gir=yes&initcwndbps=1283750&pl=14&source=youtube&ms=auh&clen=908248452&mv=m&pfa=5&mt=1508902087&expire=1508923799&hightc=yes&requiressl=yes&ipbits=0&mn=sn-5uaeznde&mm=27&id=o-AHYFq3aZY0n-Jr-aLDMCVuf3P_UEHXAlXjv8V0oHxEYM&itag=146&ei=NwXwWZ_7A9bzqQXqlagg&ip=174.108.35.211&key=yt6&keepalive=yes&dur=2857.954&lmt=1508399865663141&cmbypass=yes&aitags=142%2C143%2C144%2C145%2C146%2C161%2C222%2C223%2C224&ctier=UD&mime=video%2Fmp4&sparams=aitags%2Cclen%2Ccmbypass%2Cctier%2Cdur%2Cei%2Cgcr%2Cgir%2Chightc%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpfa%2Cpl%2Crequiressl%2Csource%2Cexpire&signature=CFE6BFD804B1D9ECC050C0A24BF21B271FA544B5.59102D4A934EA8E46253C9E9E9D56CF68B12CEE0&ratebypass=yes'
[download] Destination: Eps3.1undo.gz-Bq6ug91Qeoc.f146.mp4
[download] 100% of 866.17MiB in 01:06
[debug] Invoking downloader on u'https://r5---sn-5uaeznde.googlevideo.com/videoplayback?gcr=us&gir=yes&initcwndbps=1283750&pl=14&source=youtube&ms=auh&clen=92917250&mv=m&pfa=5&mt=1508902087&expire=1508923799&hightc=yes&requiressl=yes&ipbits=0&mn=sn-5uaeznde&mm=27&id=o-AHYFq3aZY0n-Jr-aLDMCVuf3P_UEHXAlXjv8V0oHxEYM&itag=150&ei=NwXwWZ_7A9bzqQXqlagg&ip=174.108.35.211&key=yt6&keepalive=yes&dur=2858.004&lmt=1508400112643637&cmbypass=yes&ctier=UD&mime=audio%2Fmp4&sparams=clen%2Ccmbypass%2Cctier%2Cdur%2Cei%2Cgcr%2Cgir%2Chightc%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpfa%2Cpl%2Crequiressl%2Csource%2Cexpire&signature=35125FA2AE8ED73B5BEB63A05F6D8A8EBD40AC3B.6134463D593F25C492190B1EE448D661023A9EAD&ratebypass=yes'
[download] Destination: Eps3.1undo.gz-Bq6ug91Qeoc.f150.m4a
[download] 100% of 88.61MiB in 00:06
[ffmpeg] Merging formats into "Eps3.1undo.gz-Bq6ug91Qeoc.mp4"
[debug] ffmpeg command line: ffmpeg -y -i "file:Eps3.1undo.gz-Bq6ug91Qeoc.f146.mp4" -i "file:Eps3.1undo.gz-Bq6ug91Qeoc.f150.m4a" -c copy -map "0:v:0" -map "1:a:0" "file:Eps3.1undo.gz-Bq6ug91Qeoc.temp.mp4"
Deleting original file Eps3.1undo.gz-Bq6ug91Qeoc.f146.mp4 (pass -k to keep)
Deleting original file Eps3.1undo.gz-Bq6ug91Qeoc.f150.m4a (pass -k to keep)

austinarchibald commented 6 years ago

Any update on this? Anyone figure out how to download tv.youtube.com videos?

judge2020 commented 6 years ago

It's likely proprietary DRM protected, preventing many shows from being downloaded. That would explain why youtube TV recommends not to use the original, slower chromecast (buffers for longer since it takes longer to decrypt the stream).

yan12125 commented 6 years ago

@judge2020: Could you add --write-pages and see what's in the MPD manifests (*.mpd)? youtube-dl already filters out ContentProtection-based DRM manifests.

Judging from your screenshot, I guess that's similar to HLS's SAMPLE-AES - decrypt after demux.

remitamine commented 6 years ago

i guess that it's .wvm video as the files has been downloaded with the builtin HttpFD downloader.

judge2020 commented 6 years ago

@yan12125 Here's the mpd manifest: https://ghostbin.com/paste/btj36 I removed the links containing my IP and whatnot, if needed I can provide the unedited dump.

yan12125 commented 6 years ago

Thanks! That's indeed DRM, and Google's open-sourced shaka-player can handle them (via close-sourced CDMs I guess) [1]. IIRC codes for calling CDMs are open-sourced. I remembered I've seen them in Chromium and Firefox's source trees.

[1] https://github.com/google/shaka-player/issues/358

lastb0isct commented 6 years ago

Any movement on this issue?

treesap32 commented 6 years ago

@coldwaterq @judge2020 I'm pretty new at this stuff but would like to test on additional programming on tv.youtube.com with the updated python script youtube.py. Any chance either of you could compile a new windows executable with tv.youtube.com support that I could test with?

judge2020 commented 2 years ago

If someone wants to test this it won’t be hard to find another show to download, but regardless: I doubt this ever will happen since YouTube TV doesn’t even work with software-based CDM/widevine level 3 (likely at the request of their TV channel content providers), precisely to prevent anyone from downloading the channels. Unless yt-dl is interested in embedding a 0-day widevine bypass or L1 decryption key, this isn’t going to happen.

judge2020 commented 2 years ago

https://tv.youtube.com/watch/TnPb_XFxuyo - Saturday Night Live, S47 E1= - Owen Wilson; Kacey Musgraves

YouTube TV is a paid service, but feel free.

ytdl-org / youtube-dl