BBC iPlayer doesn't see HD video options

MatthewDaws commented 4 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2019.11.28
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl -v -F https://www.bbc.co.uk/iplayer/episode/m000byxk
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-F', 'https://www.bbc.co.uk/iplayer/episode/m000byxk']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.11.28
[debug] Python version 3.7.1 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: none
[debug] Proxy map: {}
[bbc.co.uk] m000byxk: Downloading video page
[bbc.co.uk] m000byxk: Downloading playlist JSON
[bbc.co.uk] p07wgs5f: Downloading media selection XML
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc.co.uk] p07wgs5f: Downloading MPD manifest
[bbc.co.uk] p07wgs5f: Downloading MPD manifest
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[bbc.co.uk] p07wgs5f: Downloading MPD manifest
[bbc.co.uk] p07wgs5f: Downloading MPD manifest
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
[bbc.co.uk] p07wgs5f: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[info] Available formats for p07wgs5f:
format code                                                                                  extension  resolution note
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash-audio_eng_1=128000           m4a        audio only [en] DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-audio_eng_1=128000     m4a        audio only [en] DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash-audio_eng_1=128000        m4a        audio only [en] DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-audio_eng_1=128000  m4a        audio only [en] DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=827000                 mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=827000           mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=827000              mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=827000        mp4        704x396    DASH video  827k , mp4_dash container, avc3.4D401E, 25fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash-video=1570000                mp4        704x396    DASH video 1570k , mp4_dash container, avc3.64001F, 50fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_dash_https-video=1570000          mp4        704x396    DASH video 1570k , mp4_dash container, avc3.64001F, 50fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash-video=1570000             mp4        704x396    DASH video 1570k , mp4_dash container, avc3.64001F, 50fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_dash_https-video=1570000       mp4        704x396    DASH video 1570k , mp4_dash container, avc3.64001F, 50fps, video only
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_hls-1013                          mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_hls_https-1013                    mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_bidi_uk_hls-1013                            mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_bidi_uk_hls_https-1013                      mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_hls-1013                       mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_hls_https-1013                 mp4        704x396    1013k , avc1.4D401E@ 827k, 25.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_hls-1800                          mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_akamai_uk_hls_https-1800                    mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_bidi_uk_hls-1800                            mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_bidi_uk_hls_https-1800                      mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_hls-1800                       mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k
stream-uk-iptv_streaming_concrete_combined_sd_mf_limelight_uk_hls_https-1800                 mp4        704x396    1800k , avc1.64001F@1570k, 50.0fps, mp4a.40.2@128k (best)

Description

As you can see from the log above, trying to download this video only finds videos of size 704x396, i.e. "SD" quality. However, visiting that URL in a browser (latest Firefox) and pressing play shows me the HD quality video (1280x720, 5166kps as shown by right clicking in the video window on the iPlayer site). So for some reason youtube-dl is missing these higher-quality videos.

In the past I have had no problem getting the HD videos. And for a different programme, it worked fine yesterday.

MatthewDaws commented 4 years ago

I've checked and I get the same result on a different Windows machine running a different distro of Python, and on a linux virtual machine. All from the same internet connection however.

hairycactus commented 4 years ago

I get the same result on a different Windows machine running a different distro of Python, and on a linux virtual machine. All from the same internet connection however. All from the same internet connection however.

It's definitely not your internet connection. I'm encountering the same issue with the following specs on Windows:

Youtube-dl v2019.11.28
Python version v3.4.4 (CPython)
FFmpeg v4.2.1, FFprobe v4.2.1

youtube-dl is missing these higher-quality videos.

Since 26/27 Nov 2019, Youtube-dl is also not seeing any of the various lower video resolutions for the (older) main videos at BBC's video page. (I prefer 640x360 SD, since my screen size doesn't usually necessitate HD videos, unless there is tiny text in the video. )

For instance, the full range of video resolutions previously available for this recent-older video (03 Jul 2019) has gone missing, & available video resolution is now limited to just 704x396 SD.

The reason why I quoted an older video is because Youtube-dl is completely broken for all videos published since 28 Nov 2019 on BBC's main video page.

Meanwhile, all video resolutions (384x216 - 1280x720) appear to be still available for new/recent BBC Reel/Ideas/Earth etc. videos.

dirkf commented 3 years ago

With PR https://github.com/ytdl-org/youtube-dl/pull/26821, I can see 960x540 for the test programme.

It's possible that further playlists, eg with .../6/... instead of .../5/... (like the 2 currently used), could be tried to increase the range of formats returned.

dirkf commented 3 years ago

With https://github.com/ytdl-org/youtube-dl/commit/e465b25c1fb0e72b97a032220399d4a959662095, still only 704x396.

See discussion at https://github.com/ytdl-org/youtube-dl/issues/21870#issuecomment-790001948.

ytdl-org / youtube-dl