search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.78k stars 9.98k forks source link

youtube extractor raises DRM protected errors when extracting empty playlists #32408

Open amaroklopcic opened 1 year ago

amaroklopcic commented 1 year ago

Checklist

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/playlist?list=VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: cf1105f
[debug] Python 3.10.4 (CPython x86_64 64bit) - Linux-6.3.9-arch1-1-x86_64-with-glibc2.37 - OpenSSL 3.1.1 30 May 2023 - glibc 2.37
[debug] exe versions: ffmpeg 6.0, ffprobe 6.0
[debug] Proxy map: {}
[youtube:tab] VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF: Downloading webpage
[download] Downloading playlist: VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF
[youtube:tab] Downloading page 1
[youtube:tab] Downloading page 2
[youtube:tab] Downloading page 3
[youtube:tab] Downloading page 4
[youtube:tab] Downloading page 5
[youtube:tab] Downloading page 6
[youtube:tab] Downloading page 7
[youtube:tab] Downloading page 8
[youtube:tab] Downloading page 9
[youtube:tab] Downloading page 10
[youtube:tab] Downloading page 11
[youtube:tab] playlist VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF: Downloading 268 videos
[download] Downloading video 1 of 268
[youtube] t7-XtJ0O1o8: Downloading webpage
[youtube] t7-XtJ0O1o8: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
ERROR: This video is DRM protected.
Traceback (most recent call last):
  File "/home/amar/Documents/dev/ytdl-testing/.venv/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 825, in wrapper
    return func(self, *args, **kwargs)
  File "/home/amar/Documents/dev/ytdl-testing/.venv/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 846, in __extract_info
    ie_result = ie.extract(url)
  File "/home/amar/Documents/dev/ytdl-testing/.venv/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 535, in extract
    ie_result = self._real_extract(url)
  File "/home/amar/Documents/dev/ytdl-testing/.venv/lib/python3.10/site-packages/youtube_dl/extractor/youtube.py", line 2100, in _real_extract
    raise ExtractorError(
youtube_dl.utils.ExtractorError: This video is DRM protected.

Description

Hey guys :)

I didn't check checklist items 1 or 3 since they don't seem applicable to this small issue. The extractor is not broken and works and in this instance there are no actual playable songs/videos, since I'm trying to "extract" an empty playlist.

So this is more so to report potentially incorrect exception messages (since there isn't any DRM protected content in the playlist). It looks like the YouTube extractor makes an incorrect assumption and throws a DownloadError: ERROR: This video is DRM protected when trying to extract an empty public playlist.

Here's a link to the empty public playlist for reference: https://www.youtube.com/playlist?list=VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF

Apologies if this issue makes more sense as a feature request, I wasn't sure if this was intended behavior or not so thought I would post it as a bug report.

dirkf commented 1 year ago

All the checklist items are always relevant!

... there are no actual playable songs/videos, since I'm trying to "extract" an empty playlist.

Not empty for me, nor for you since it had 268 items. Now I'm seeing 278 items.

This doesn't happen for me with the latest master code. The code that generates the DRM message is at l.2107-8 in my version and it depends on (i) no formats being found (ii) the streamingData.licenseInfos member of the player response JSON being set. So you may want to update from the latest master or from the new nightly build repo (#31585).

The problem video t7-XtJ0O1o8 gives me this (UK):

...
[youtube] t7-XtJ0O1o8: Downloading webpage
ERROR: Video unavailable
The uploader has not made this video available in your country
...
gamer191 commented 1 year ago

visiting https://www.youtube.com/playlist?list=VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF makes youtube show a weird home page view As for https://www.youtube.com/watch?v=t7-XtJ0O1o8, that's a free with ads video, and afaik they're always DRM protected :(

dirkf commented 1 year ago

There's also https://github.com/yt-dlp/yt-dlp/pull/7396.

gamer191 commented 1 year ago

that issue only affects iOS-exclusive formats afaik

Does youtube-dl implement the android and ios clients, or only the web one?

dirkf commented 1 year ago

Currently WEB and TVHTML5_SIMPLY_EMBEDDED_PLAYER (for age-gated content) are used. But the issue solved in the linked PR could apply to manifests in general, not just from YT, I think.

gamer191 commented 1 year ago

@dirkf I kinda agree, but like pukkandan said in the PR description:

In theory, it is possible for these types of master manifests to be used by other sites as well. But changing the DRM detection in core code directly will cause all DRM manifests to only be detected as "Maybe DRM", which is undesirable. So unless we find such cases in the wild, I am opting to make this change only for youtube and leave the general parsing as-is.

Also, that PR isn't really suitable to be enabled outside the youtube extractor, because --check-formats currently only detects Apple fairplay DRM. That's why it's only enabled in the youtube extractor for the HLS protocol. I'm not sure if Widevine/Playready can be included in HLS streams, but afaik Youtube only uses HLS for iOS formats, more reason that that code likely isn't ready for other extractors

Vangelis66 commented 1 year ago

As for https://www.youtube.com/watch?v=t7-XtJ0O1o8 , that's a free with ads video, and afaik they're always DRM protected :(

I can confirm, by using a) a US IP address (via VPN) b) latest yt-dlp-nightly and issuing: 

yt-dlp --allow-u -F "t7-XtJ0O1o8"
Result ```console WARNING: You have asked for UNPLAYABLE formats to be listed/downloaded. This is a developer option intended for debugging. If you experience any issues while using this option, DO NOT open a bug report [youtube+AGB+NSIG] Extracting URL: t7-XtJ0O1o8 [youtube+AGB+NSIG] t7-XtJ0O1o8: Downloading webpage [youtube+AGB+NSIG] t7-XtJ0O1o8: Downloading ios player API JSON [youtube+AGB+NSIG] t7-XtJ0O1o8: Downloading android player API JSON [youtube+AGB+NSIG] t7-XtJ0O1o8: Downloading m3u8 information [youtube+AGB+NSIG] t7-XtJ0O1o8: Downloading MPD manifest WARNING: [youtube+AGB+NSIG] Failed to download MPD manifest: HTTP Error 404: Not Found [info] Available formats for t7-XtJ0O1o8: ID EXT RESOLUTION FPS CH | FILESIZE TBR PROTO | VCODEC VBR ACODEC ABR ASR MORE INFO ------------------------------------------------------------------------------------------------------------------------------------------------------- 293 mp4 audio only | m3u8 | audio only unknown [en] DRM, English - descriptive 294 mp4 audio only | m3u8 | audio only unknown [en] DRM, English - descriptive 295 mp4 audio only | m3u8 | audio only unknown [en] DRM, English - descriptive 688-0 m4a audio only 2 | 32.87MiB 49k https | audio only mp4a.40.5 49k 22k [en-desc] DRM, English descriptive, low, m4a_dash 148-0 m4a audio only 2 | 33.82MiB 50k https | audio only mp4a.40.5 50k 22k [en-desc] DRM, English descriptive, low, m4a_dash 688-1 m4a audio only 2 | 32.87MiB 49k https | audio only mp4a.40.5 49k 22k [en] DRM, English original (default), low, m4a_dash 148-1 m4a audio only 2 | 33.82MiB 50k https | audio only mp4a.40.5 50k 22k [en] DRM, English original (default), low, m4a_dash 350-0 webm audio only 2 | 35.98MiB 53k https | audio only opus 53k 48k [en-desc] DRM, English descriptive, low, webm_dash 351-0 webm audio only 2 | 45.86MiB 68k https | audio only opus 68k 48k [en-desc] DRM, English descriptive, low, webm_dash 350-1 webm audio only 2 | 35.64MiB 53k https | audio only opus 53k 48k [en] DRM, English original (default), low, webm_dash 351-1 webm audio only 2 | 45.75MiB 68k https | audio only opus 68k 48k [en] DRM, English original (default), low, webm_dash 689-0 m4a audio only 2 | 87.24MiB 129k https | audio only mp4a.40.2 129k 44k [en-desc] DRM, English descriptive, medium, m4a_dash 149-0 m4a audio only 2 | 89.11MiB 132k https | audio only mp4a.40.2 132k 44k [en-desc] DRM, English descriptive, medium, m4a_dash 689-1 m4a audio only 2 | 87.24MiB 129k https | audio only mp4a.40.2 129k 44k [en] DRM, English original (default), medium, m4a_dash 149-1 m4a audio only 2 | 89.11MiB 132k https | audio only mp4a.40.2 132k 44k [en] DRM, English original (default), medium, m4a_dash 352-0 webm audio only 2 | 87.22MiB 129k https | audio only opus 129k 48k [en-desc] DRM, English descriptive, medium, webm_dash 352-1 webm audio only 2 | 87.21MiB 129k https | audio only opus 129k 48k [en] DRM, English original (default), medium, webm_dash 690-0 m4a audio only 2 | 173.48MiB 257k https | audio only mp4a.40.2 257k 44k [en-desc] DRM, English descriptive, high, m4a_dash 150-0 m4a audio only 2 | 175.36MiB 260k https | audio only mp4a.40.2 260k 44k [en-desc] DRM, English descriptive, high, m4a_dash 690-1 m4a audio only 2 | 173.49MiB 257k https | audio only mp4a.40.2 257k 44k [en] DRM, English original (default), high, m4a_dash 150-1 m4a audio only 2 | 175.36MiB 260k https | audio only mp4a.40.2 260k 44k [en] DRM, English original (default), high, m4a_dash 326-0 m4a audio only 6 | 259.33MiB 385k https | audio only dtse 385k 48k [en-desc] DRM, English descriptive, high, m4a_dash 326-1 m4a audio only 6 | 259.33MiB 385k https | audio only dtse 385k 48k [en] DRM, English original (default), high, m4a_dash 381-0 m4a audio only 6 | 260.17MiB 386k https | audio only ac-3 386k 48k [en-desc] DRM, English descriptive, high, m4a_dash 381-1 m4a audio only 6 | 260.17MiB 386k https | audio only ac-3 386k 48k [en] DRM, English original (default), high, m4a_dash 691-0 m4a audio only 6 | 258.80MiB 384k https | audio only ec-3 384k 48k [en-desc] DRM, English descriptive, high, m4a_dash 329-0 m4a audio only 6 | 260.17MiB 386k https | audio only ec-3 386k 48k [en-desc] DRM, English descriptive, high, m4a_dash 691-1 m4a audio only 6 | 258.81MiB 384k https | audio only ec-3 384k 48k [en] DRM, English original (default), high, m4a_dash 329-1 m4a audio only 6 | 260.17MiB 386k https | audio only ec-3 386k 48k [en] DRM, English original (default), high, m4a_dash 261-0 m4a audio only 6 | 263.36MiB 391k https | audio only mp4a.40.2 391k 48k [en-desc] DRM, English descriptive, high, m4a_dash 261-1 m4a audio only 6 | 263.37MiB 391k https | audio only mp4a.40.2 391k 48k [en] DRM, English original (default), high, m4a_dash 281 mp4 256x144 24 | ~152.19MiB 221k m3u8 | avc1.4D400C 221k video only DRM 161 mp4 256x144 24 | 77.38MiB 115k https | avc1.4d400c 115k video only DRM, 144p, mp4_dash 282 mp4 426x240 24 | ~259.58MiB 376k m3u8 | avc1.4D4015 376k video only DRM 142 mp4 426x240 24 | 168.00MiB 249k https | avc1.4d4015 249k video only DRM, 240p, mp4_dash 279 webm 426x240 24 | 82.48MiB 122k https | vp9 122k video only DRM, 240p, webm_dash 647 mp4 426x240 24 | 82.98MiB 123k https | vp09.00.20.08 123k video only DRM, 240p, mp4_dash 283 mp4 640x360 24 | ~598.59MiB 868k m3u8 | avc1.4D401E 868k video only DRM 143 mp4 640x360 24 | 243.11MiB 361k https | avc1.4d401e 361k video only DRM, 360p, mp4_dash 648 mp4 640x360 24 | 149.99MiB 223k https | vp09.00.21.08 223k video only DRM, 360p, mp4_dash 280 webm 640x360 24 | 168.81MiB 251k https | vp9 251k video only DRM, 360p, webm_dash 284 mp4 854x480 24 | ~ 1.04GiB 1543k m3u8 | avc1.4D401E 1543k video only DRM 285 mp4 854x480 24 | ~ 2.01GiB 2978k m3u8 | avc1.4D401E 2978k video only DRM 286 mp4 854x480 24 | ~ 2.63GiB 3908k m3u8 | avc1.4D401E 3908k video only DRM 144 mp4 854x480 24 | 699.47MiB 1038k https | avc1.4d401e 1038k video only DRM, 480p, mp4_dash 222 mp4 854x480 24 | 1.10GiB 1665k https | avc1.4d401e 1665k video only DRM, 480p, mp4_dash 223 mp4 854x480 24 | 1.78GiB 2713k https | avc1.4d401e 2713k video only DRM, 480p, mp4_dash 649 mp4 854x480 24 | 253.23MiB 376k https | vp09.00.30.08 376k video only DRM, 480p, mp4_dash 650 mp4 854x480 24 | 293.09MiB 435k https | vp09.00.30.08 435k video only DRM, 480p, mp4_dash 317 webm 854x480 24 | 339.40MiB 504k https | vp9 504k video only DRM, 480p, webm_dash 651 mp4 854x480 24 | 362.42MiB 538k https | vp09.00.30.08 538k video only DRM, 480p, mp4_dash 318 webm 854x480 24 | 596.76MiB 886k https | vp9 886k video only DRM, 480p, webm_dash 273 webm 854x480 24 | 1.61GiB 2445k https | vp9 2445k video only DRM, 480p, webm_dash 287 mp4 1280x720 24 | ~ 2.73GiB 4056k m3u8 | avc1.4D401F 4056k video only DRM 288 mp4 1280x720 24 | ~ 2.77GiB 4118k m3u8 | avc1.4D401F 4118k video only DRM 289 mp4 1280x720 24 | ~ 4.24GiB 6288k m3u8 | avc1.4D401F 6288k video only DRM 145 mp4 1280x720 24 | 1.49GiB 2257k https | avc1.4d401f 2257k video only DRM, 720p, mp4_dash 224 mp4 1280x720 24 | 2.25GiB 3419k https | avc1.4d401f 3419k video only DRM, 720p, mp4_dash 225 mp4 1280x720 24 | 3.70GiB 5629k https | avc1.4d401f 5629k video only DRM, 720p, mp4_dash 652 mp4 1280x720 24 | 530.94MiB 788k https | vp09.00.31.08 788k video only DRM, 720p, mp4_dash 653 mp4 1280x720 24 | 649.01MiB 963k https | vp09.00.31.08 963k video only DRM, 720p, mp4_dash 654 mp4 1280x720 24 | 687.76MiB 1021k https | vp09.00.31.08 1021k video only DRM, 720p, mp4_dash 274 webm 1280x720 24 | 694.41MiB 1031k https | vp9 1031k video only DRM, 720p, webm_dash 357 webm 1280x720 24 | 1.21GiB 1840k https | vp9 1840k video only DRM, 720p, webm_dash 358 webm 1280x720 24 | 2.25GiB 3425k https | vp9 3425k video only DRM, 720p, webm_dash 291 mp4 1920x1080 24 | ~ 4.60GiB 6834k m3u8 | avc1.640028 6834k video only DRM 290 mp4 1920x1080 24 | ~ 5.54GiB 8223k m3u8 | avc1.640028 8223k video only DRM 292 mp4 1920x1080 24 | ~ 6.01GiB 8914k m3u8 | avc1.640028 8914k video only DRM 146 mp4 1920x1080 24 | 2.83GiB 4302k https | avc1.640028 4302k video only DRM, 1080p, mp4_dash 226 mp4 1920x1080 24 | 3.76GiB 5716k https | avc1.640028 5716k video only DRM, 1080p, mp4_dash 227 mp4 1920x1080 24 | 5.24GiB 7969k https | avc1.640028 7969k video only DRM, 1080p, mp4_dash 275 webm 1920x1080 24 | 1.26GiB 1919k https | vp9 1919k video only DRM, 1080p, webm_dash 657 mp4 1920x1080 24 | 1.27GiB 1924k https | vp09.00.40.08 1924k video only DRM, 1080p, mp4_dash 658 mp4 1920x1080 24 | 2.05GiB 3111k https | vp09.00.40.08 3111k video only DRM, 1080p, mp4_dash 659 mp4 1920x1080 24 | 2.37GiB 3607k https | vp09.00.40.08 3607k video only DRM, 1080p, mp4_dash 360 webm 1920x1080 24 | 3.46GiB 5261k https | vp9 5261k video only DRM, 1080p, webm_dash ```

FWIW, I don't think the original report "can hold any water" 😉 ... I did load the linked playlist https://www.youtube.com/playlist?list=VLPLdZmvfk6Qqb7PFtsqGqNQ4804xSTX2dlF in my browser (with a US IP address) and, as pointed out already, it isn't "empty", the same is true as far as yt-dl[p] is concerned... The WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found is there on yt-dlp, too, but probably refers to a 404 MPD corresponding to a specific yt-dl client (?) ... The first item of that playlist (t7-XtJ0O1o8) indeed turns out to be DRM-only, so YoutubeIE is just printing a correct ERROR report...