[HGTVDe] No video formats found

knalp commented 1 year ago

Checklist

[X] I'm reporting a broken site support
[X] I've verified that I'm running youtube-dl version 2021.12.17
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl.exe --verbose https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.22621
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev, ffprobe 4.4-full_build-www.gyan.dev
[debug] Proxy map: {}
[debug] Using fake IP 53.143.235.23 (DE) as X-Forwarded-For.
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading token
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading JSON metadata
[HGTVDe] 4271856: Downloading JSON metadata
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading m3u8 information
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading MPD manifest
WARNING: [HGTVDe] Unknown MIME type image/jpeg in DASH manifest
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 815, in wrapper
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 836, in __extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 534, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\dplay.py", line 369, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\dplay.py", line 244, in _get_disco_api_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 1374, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

...could not download a video from HGTV. Please see the complete command with the debug output above.

Vangelis66 commented 1 year ago

... "downstream" seem to handle the URL provided in OP as expected 😜 :

yt-dlp -vF "https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm" =>

[debug] Command-line config: ['--ffmpeg-location', '..\\FFmpeg', '--downloader-args', 'ffmpeg:-v 8 -stats', '-vF', 'https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2023.02.14 [8b37c58f8] (win_x86_exe)
[debug] Python 3.7.16 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1t  7 Feb 2023)
[debug] exe versions: ffmpeg n5.2-dev-2245-N-109649-gab8cde6 (setts), ffprobe n5.2-dev-2245-N-109649-gab8cde6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Extractor Plugins: AGB+NSIG (YoutubeIE)
[debug] Plugin directories: ['<redacted>\\yt-dlp\\yt-dlp-plugins\\YTAgeGateBypass.zip\\yt_dlp_plugins', '<redacted>\\yt-dlp\\yt-dlp-plugins\\YTNSigProxy.zip\\yt_dlp_plugins']
[debug] Loaded 1767 extractors
[HGTVDe] Extracting URL: https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm
[debug] Using fake IP 53.187.54.168 (DE) as X-Forwarded-For
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading token
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading JSON metadata
[HGTVDe] 4271856: Downloading JSON metadata
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading MPD manifest
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 4271856:
ID                         EXT   RESOLUTION FPS |   FILESIZE   TBR PROTO | VCODEC        VBR ACODEC     MORE INFO
-------------------------------------------------------------------------------------------------------------------------------
dash-thumbnails            mhtml unknown        |                  mhtml | images                       DASH storyboards (jpeg)
hls-audio-aacl-125-English mp4   audio only     |                  m3u8  | audio only        unknown    [en] English
hls-304                    mp4   640x360     25 | ~ 98.79MiB  304k m3u8  | avc1.4D4016  304k video only
hls-413                    mp4   640x360     25 | ~134.20MiB  413k m3u8  | avc1.4D4016  413k video only
hls-645                    mp4   960x540     25 | ~209.59MiB  645k m3u8  | avc1.4D401F  645k video only
hls-960                    mp4   1280x720    25 | ~311.95MiB  960k m3u8  | avc1.4D401F  960k video only
hls-1645                   mp4   1280x720    25 | ~534.54MiB 1645k m3u8  | avc1.4D401F 1645k video only
hls-3021                   mp4   1920x1080   25 | ~981.68MiB 3021k m3u8  | avc1.4D4028 3021k video only
hls-4885                   mp4   1920x1080   25 | ~  1.55GiB 4885k m3u8  | avc1.4D4028 4885k video only
hls-8081                   mp4   1920x1080   25 | ~  2.56GiB 8081k m3u8  | avc1.4D4028 8081k video only

... so I suspect a backport to youtube-dl would be needed... FWIW, I see no DASH media streams returned, so whatever the issue is with the MPD (DASH) manifest (that causes a WARNING in youtube-dl), the m3u8 (HLS) manifest is parsed OK by yt-dlp ...

dirkf commented 1 year ago

Are these formats playable?

Both programs get these manifests:

{'type': 'dash',
  'url': 'https://dplaydk-prod-vod.akamaized.net/3ccb28e5-ce90-4926-a998-2f2cb9456fe4/dash_widevinePlayready_fmp4/hdntl=exp=1677437647~acl=/3ccb28e5-ce90-4926-a998-2f2cb9456fe4/dash_widevinePlayready_fmp4/*~data=hdntl~hmac=6ac174be589a5be1361e64fcb87f137610fe802361d3b2575d5dea635275efec/master.mpd'},
 {'type': 'hls',
  'url': 'https://dplaydk.media-edge.prod-vod.h264.io/3ccb28e5-ce90-4926-a998-2f2cb9456fe4/hls_fairPlay_ts/edge-cache-token=Expires=1677437647&KeyName=media-edge-key-prod-eu-661a302&Signature=PyyNvCwfVBtnnA8YRmlXTO-LfrOGuzJZPcgMBdK0HJHophLxqqZdRFKPq701-hoKMQeh_jUmWFaaIBCRnCTyCA/master.m3u8'}

They look a bit DRM-ish to me. Anyhow, there is a difference in the manifest processing: yt-dl skips any DRM-ish formats while yt-dlp just remembers:

--- youtube_dl/extractor/common.py
+++ yt_dlp/extractor/common.py
-        if '#EXT-X-FAXS-CM:' in m3u8_doc:  # Adobe Flash Access
-            return []
-
-        if re.search(r'#EXT-X-SESSION-KEY:.*?URI="skd://', m3u8_doc):  # Apple FairPlay
-            return []
+        has_drm = re.search('|'.join([
+            r'#EXT-X-FAXS-CM:',  # Adobe Flash Access
+            r'#EXT-X-(?:SESSION-)?KEY:.*?URI="skd://',  # Apple FairPlay
+        ]), m3u8_doc)

yt-dlp doesn't set a has_drm field in the HLS formats, although the manifest has this:

#EXTM3U
#EXT-X-VERSION:4
## Created with Unified Streaming Platform (version=1.11.17-26567)
#EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://eyJDb250ZW50SWQiOiIwMDAwMDNlOS1jNzkzLTZiZTktMjUxZC1lODhkMGUz'

Although has_drm is set as above, the value is only included in the HLS formats for discontinuous media. Hence yt-dlp produces a set of HLS formats while yt-dl doesn't.

Vangelis66 commented 1 year ago

Are these formats playable?

Once more, you are correct, sir 😉 👍 ...

Although --allow-u wasn't issued in the yt-dlp command I used in my previous post, it comes, in the end, that all streams detected by yt-dlp are, indeed, and alas :rage:, behind full-blown DRM (Apple's FairPlay):

yt-dlp --check-formats "https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm" => 

[HGTVDe] Extracting URL: https://de.hgtv.com/sendungen/unser-traum-vom-schloss/die-ruhe-vor-dem-sturm
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading token
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading JSON metadata
[HGTVDe] 4271856: Downloading JSON metadata
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading MPD manifest
[HGTVDe] unser-traum-vom-schloss/die-ruhe-vor-dem-sturm: Downloading m3u8 information
[info] Testing format hls-8081
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-8081. Skipping...
[info] Testing format hls-4885
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-4885. Skipping...
[info] Testing format hls-3021
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-3021. Skipping...
[info] Testing format hls-1645
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-1645. Skipping...
[info] Testing format hls-960
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-960. Skipping...
[info] Testing format hls-645
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-645. Skipping...
[info] Testing format hls-413
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-413. Skipping...
[info] Testing format hls-304
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-304. Skipping...
[info] Testing format hls-audio-aacl-125-English
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format
[info] Unable to download format hls-audio-aacl-125-English. Skipping...
ERROR: [HGTVDe] 4271856: Requested format is not available. Use --list-formats for a list of available formats

@knalp : Tough luck... 😿

dirkf commented 1 year ago

Is this the expected behaviour for yt-dlp, @pukkandan ?

pukkandan commented 1 year ago

No. Thanks for the ping. Fixed now

dirkf commented 1 year ago

So, DRM. Not all shows are affected: the test show is OK. Maybe all future content will be afflicted, though.

ytdl-org / youtube-dl