[Broken] All CBC Gem URLs

makew0rld commented 3 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running yt-dlp version 2021.09.02
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

Error when downloading live content: (will need a different URL once this stream is no longer live. Live stream links can be obtained from https://gem.cbc.ca but will be geo-blocked to Canada)

➤ yt-dlp -F https://gem.cbc.ca/live/1943125059642 -v
[debug] Command-line config: ['-F', 'https://gem.cbc.ca/live/1943125059642', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.09.02 
[debug] Python version 3.9.6 (CPython 64bit) - Linux-5.13.13-arch1-1-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: mutagen, pycryptodome, sqlite, websockets
[debug] Proxy map: {}
[debug] Using fake IP 99.246.103.23 (CA) as X-Forwarded-For
[debug] [cbc.ca:watch] Extracting URL: https://gem.cbc.ca/live/1943125059642
[cbc.ca:watch] 1943125059642: Downloading XML
[cbc.ca:watch] 1943125059642: Downloading XML
ERROR: [cbc.ca:watch] 1943125059642: cbc.ca:watch said: We are sorry, but the selected video is currently unavailable.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/cbc.py", line 436, in _real_extract
    rss = self._call_api('web/browse/' + video_id, video_id)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/cbc.py", line 261, in _call_api
    raise ExtractorError('%s said: %s' % (self.IE_NAME, error_message))
yt_dlp.utils.ExtractorError: cbc.ca:watch said: We are sorry, but the selected video is currently unavailable.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/cbc.py", line 436, in _real_extract
    rss = self._call_api('web/browse/' + video_id, video_id)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/cbc.py", line 261, in _call_api
    raise ExtractorError('%s said: %s' % (self.IE_NAME, error_message))
yt_dlp.utils.ExtractorError: cbc.ca:watch said: We are sorry, but the selected video is currently unavailable.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1214, in wrapper
    return func(self, *args, **kwargs)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1239, in __extract_info
    ie_result = ie.extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 600, in extract
    raise ExtractorError(
yt_dlp.utils.ExtractorError: [cbc.ca:watch] 1943125059642: cbc.ca:watch said: We are sorry, but the selected video is currently unavailable.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

Error when downloading non-live news content:

➤ yt-dlp -F https://gem.cbc.ca/media/the-national/s01e202198 -v 
[debug] Command-line config: ['-F', 'https://gem.cbc.ca/media/the-national/s01e202198', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.09.02 
[debug] Python version 3.9.6 (CPython 64bit) - Linux-5.13.13-arch1-1-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: mutagen, pycryptodome, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://gem.cbc.ca/media/the-national/s01e202198
[generic] s01e202198: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] s01e202198: Downloading webpage
[generic] s01e202198: Extracting information
ERROR: [generic] Unsupported URL: https://gem.cbc.ca/media/the-national/s01e202198
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/generic.py", line 3713, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://gem.cbc.ca/media/the-national/s01e202198
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/generic.py", line 3713, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://gem.cbc.ca/media/the-national/s01e202198

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1214, in wrapper
    return func(self, *args, **kwargs)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1239, in __extract_info
    ie_result = ie.extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 600, in extract
    raise ExtractorError(
yt_dlp.utils.ExtractorError: [generic] Unsupported URL: https://gem.cbc.ca/media/the-national/s01e202198

Error when downloading a TV show that does not require login:

➤ yt-dlp -F https://gem.cbc.ca/media/schitts-creek/s06e01 -v
[debug] Command-line config: ['-F', 'https://gem.cbc.ca/media/schitts-creek/s06e01', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.09.02 
[debug] Python version 3.9.6 (CPython 64bit) - Linux-5.13.13-arch1-1-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4, ffprobe 4.4, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: mutagen, pycryptodome, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://gem.cbc.ca/media/schitts-creek/s06e01
[generic] s06e01: Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] s06e01: Downloading webpage
[generic] s06e01: Extracting information
ERROR: [generic] Unsupported URL: https://gem.cbc.ca/media/schitts-creek/s06e01
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/generic.py", line 3713, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://gem.cbc.ca/media/schitts-creek/s06e01
Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 584, in extract
    ie_result = self._real_extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/generic.py", line 3713, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://gem.cbc.ca/media/schitts-creek/s06e01

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1214, in wrapper
    return func(self, *args, **kwargs)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 1239, in __extract_info
    ie_result = ie.extract(url)
  File "/home/makeworld/.local/pipx/venvs/yt-dlp/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 600, in extract
    raise ExtractorError(
yt_dlp.utils.ExtractorError: [generic] Unsupported URL: https://gem.cbc.ca/media/schitts-creek/s06e01

I assume additional functionality will be fixed when this is fixed, such as news and TV show playlists, as well as downloading logged in content, which was all working on youtube-dl previously. youtube-dl still has the same issue now though.

Description

None of the videos are downloading, and additionally all the non-live content is not being recognized as CBC, so the generic extractor is being used instead, incorrectly.

Thanks a lot for maintaining yt-dlp!

jpenguin commented 3 years ago

Is not because geo-blocking, gem changed url schemes. An old url I have still works, but you can't seem to get it via GEM anymore. I did trhis test on a Toronto VPN

URL retrieved a couple weeks ago-- https://gem.cbc.ca/media/heartland/season-14/episode-1/38e815a-0137fd5e960 URL GEM gives now-- https://gem.cbc.ca/media/heartland/s14e01

ron-s commented 3 years ago

Looks like CBC just overhauled their streaming site. They no longer offer 1080p unless you use their app https://cbchelp.cbc.ca/hc/en-ca/articles/115003511913-Is-HD-high-definition-video-available-Adjusting-Gem-streaming-quality-

pukkandan commented 3 years ago

The geo-blocked tag does not mean your issue is caused due to geo-blocking, just that I am unable to access the site. If you can give details of a proxy that works, then I can investigate further

jpenguin commented 3 years ago

I use vpervpn to access cbc, it does only seem to offer up to 720p for videos now. I guess you could use www.vpngate.net for testing though

makew0rld commented 3 years ago

The fact they don't offer 1080p anymore is irrelevant to my issue. It seems like @jpenguin is right and the URLs that have changed. I'm not sure what this means in terms of the backend, but I might be able to look into it.

yt-dlp can also get rid of the CBCOlympicsIE class, as olympics.cbc.ca is no longer used by CBC and the links don't work anymore.

makew0rld commented 3 years ago

Looks like the backend is totally different as well.

Here's what my browser does, which is easily repeatable with curl as you can see below. The URL is https://gem.cbc.ca/media/schitts-creek/s06e01


# URL for each episode that returns JSON with info.
# It contains all the metadata you'd need, like title, description, thumbnail, season, episode, duration, age rating, and category.

curl -sS https://services.radio-canada.ca/ott/cbc-api/v2/assets/schitts-creek/s06e01 | jq -r '.playSession.url'
https://services.radio-canada.ca/media/validation/v2?appCode=gem&idMedia=163782&manifestType=desktop&output=json&tech=hls

# Now retrieve that URL, and pick out the URL within the JSON it gives

curl -sS https://services.radio-canada.ca/media/validation/v2\?appCode\=gem\&idMedia\=163782\&manifestType\=desktop\&output\=json\&tech\=hls | jq -r '.url'
https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631727266~exp=1631727386~acl=*~hmac=501f0c05c526ca545d3524b07a919febcb7bac9c1cb9771e00d732c7e1be54de

That final URL returns an m3u8 file that looks like this:

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="English",LANGUAGE="eng",DEFAULT=YES,AUTOSELECT=YES,URI="QualityLevels(192000)/Manifest(audio_eng_aacl,format=m3u8-aapl,filter=desktop)"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="English (Descriptive)",LANGUAGE="eng",AUTOSELECT=YES,CHARACTERISTICS="public.accessibility.describes-video",URI="QualityLevels(192000)/Manifest(audiodv_eng_aacl,format=m3u8-aapl,filter=desktop)"
#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="CC",LANGUAGE="en",NAME="English",DEFAULT=YES,AUTOSELECT=YES,INSTREAM-ID="CC1"
#EXT-X-STREAM-INF:BANDWIDTH=2767619,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(2500074)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=2767619,RESOLUTION=1280x720,CODECS="avc1.4d401f",URI="QualityLevels(2500074)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=3687864,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(3400509)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=3687864,RESOLUTION=1280x720,CODECS="avc1.4d401f",URI="QualityLevels(3400509)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=2052428,RESOLUTION=960x540,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(1800278)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=2052428,RESOLUTION=960x540,CODECS="avc1.4d401f",URI="QualityLevels(1800278)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=1234564,RESOLUTION=640x360,CODECS="avc1.42c01e,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(1000020)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=1234564,RESOLUTION=640x360,CODECS="avc1.42c01e",URI="QualityLevels(1000020)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=825755,RESOLUTION=640x360,CODECS="avc1.42c01e,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(600011)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=825755,RESOLUTION=640x360,CODECS="avc1.42c01e",URI="QualityLevels(600011)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=621427,RESOLUTION=416x234,CODECS="avc1.42c00d,mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(400082)/Manifest(video,format=m3u8-aapl,filter=desktop)
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=621427,RESOLUTION=416x234,CODECS="avc1.42c00d",URI="QualityLevels(400082)/Manifest(video,format=m3u8-aapl,filter=desktop,type=keyframes)"
#EXT-X-STREAM-INF:BANDWIDTH=204384,CODECS="mp4a.40.2",AUDIO="audio",CLOSED-CAPTIONS="CC"
QualityLevels(192000)/Manifest(audiodv_eng_aacl,format=m3u8-aapl,filter=desktop)

As you can see, all the streams are there, with information on each of them. I couldn't get VLC to play from the m3u8 URL, but ffplay (an ffmpeg tool) handled it just fine.

I'm working on a PR for this. Will update the thread when it's published.

I haven't looked into live streams yet, but will update if I do. Same with getting subtitles.

makew0rld commented 3 years ago

@pukkandan @Ashish0804 I'm getting a "Requested format is not available" error with my PR code so far. Can you help? Should I open a draft PR or regular PR instead of discussing it here?

Full error log, with extracted formats printed

``` python test/test_download.py TestDownload.test_CBCGem [debug] [cbcgem] Extracting URL: https://gem.cbc.ca/media/schitts-creek/s06e01 [cbcgem] schitts-creek/s06e01: Downloading JSON metadata [cbcgem] schitts-creek/s06e01: Downloading JSON metadata [cbcgem] schitts-creek/s06e01: Downloading m3u8 information [{'ext': None, 'format_id': 'audio-English', 'format_index': None, 'format_note': 'English', 'language': 'eng', 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(192000)/Manifest(audio_eng_aacl,format=m3u8-aapl,filter=desktop)', 'vcodec': 'none'}, {'ext': None, 'format_id': 'audio-English (Descriptive)', 'format_index': None, 'format_note': 'English (Descriptive)', 'language': 'eng', 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(192000)/Manifest(audiodv_eng_aacl,format=m3u8-aapl,filter=desktop)', 'vcodec': 'none'}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '2767', 'format_index': None, 'fps': None, 'height': 720, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 2767.619, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(2500074)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.4d401f', 'width': 1280}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '3687', 'format_index': None, 'fps': None, 'height': 720, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 3687.864, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(3400509)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.4d401f', 'width': 1280}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '2052', 'format_index': None, 'fps': None, 'height': 540, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 2052.428, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(1800278)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.4d401f', 'width': 960}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '1234', 'format_index': None, 'fps': None, 'height': 360, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 1234.564, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(1000020)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.42c01e', 'width': 640}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '825', 'format_index': None, 'fps': None, 'height': 360, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 825.755, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(600011)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.42c01e', 'width': 640}, {'acodec': 'none', 'ext': 'mp4', 'format_id': '621', 'format_index': None, 'fps': None, 'height': 234, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 621.427, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(400082)/Manifest(video,format=m3u8-aapl,filter=desktop)', 'vcodec': 'avc1.42c00d', 'width': 416}, {'acodec': 'mp4a.40.2', 'ext': 'm4a', 'format_id': '204', 'format_index': None, 'fps': None, 'manifest_url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/desktop_master.m3u8?hdnea=st=1631759983~exp=1631760103~acl=*~hmac=043a2b6c484517ce97d6d092acffe4ac16f1679ba54eb4031d8f37ce76b53509', 'preference': None, 'protocol': 'm3u8_native', 'quality': None, 'tbr': 204.384, 'url': 'https://cbcrcott-gem.akamaized.net/293c48a1-3e1b-4818-b65e-db1445af5674/CBC_SCHITTS_CREEK_S06E01.ism/QualityLevels(192000)/Manifest(audiodv_eng_aacl,format=m3u8-aapl,filter=desktop)', 'vcodec': 'none'}] [debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id ERROR: [cbcgem] schitts-creek/s06e01: Requested format is not available Traceback (most recent call last): File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1218, in wrapper return func(self, *args, **kwargs) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1256, in __extract_info return self.process_ie_result(ie_result, download, extra_info) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1306, in process_ie_result ie_result = self.process_video_result(ie_result, download=download) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 2316, in process_video_result raise ExtractorError('Requested format is not available', expected=True, yt_dlp.utils.ExtractorError: [cbcgem] schitts-creek/s06e01: Requested format is not available E ====================================================================== ERROR: test_CBCGem (__main__.TestDownload): ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1218, in wrapper return func(self, *args, **kwargs) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1256, in __extract_info return self.process_ie_result(ie_result, download, extra_info) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1306, in process_ie_result ie_result = self.process_video_result(ie_result, download=download) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 2316, in process_video_result raise ExtractorError('Requested format is not available', expected=True, yt_dlp.utils.ExtractorError: [cbcgem] schitts-creek/s06e01: Requested format is not available During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/makeworld/Desktop/Coding/Python/yt-dlp/test/test_download.py", line 170, in test_template res_dict = ydl.extract_info( File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1210, in extract_info return self.__extract_info(url, self.get_info_extractor(ie_key), download, extra_info, process) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 1227, in wrapper self.report_error(compat_str(e), e.format_traceback()) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 818, in report_error self.trouble(error_message, tb) File "/home/makeworld/Desktop/Coding/Python/yt-dlp/yt_dlp/YoutubeDL.py", line 783, in trouble raise DownloadError(message, exc_info) yt_dlp.utils.DownloadError: ERROR: [cbcgem] schitts-creek/s06e01: Requested format is not available ---------------------------------------------------------------------- Ran 1 test in 0.897s FAILED (errors=1) ```

The in-progress code is available here. An example of the retrieved m3u8 file is in my comment above. Both ffplay and mpv can play the m3u8 file fine, so yt-dlp should be able to get it. I think the issue might be that none of the output formats are recognized as having both audio and video together, even though the m3u8 file indicates they're available. I think it might be because of these lines (not sure though):

https://github.com/yt-dlp/yt-dlp/blob/0001fcb586c3ab297cd48c77ddd6f5d40546dac4/yt_dlp/extractor/common.py#L2198-L2201

Ashish0804 commented 3 years ago

try adding

'params': {'format': 'bv',}

To the test

makew0rld commented 3 years ago

Thanks, that worked. I will submit my PR when it's done.

makew0rld commented 3 years ago

PR submitted: #1013

mshiels commented 2 years ago

Should this support Subtitles?? I don't seem to be getting them anymore but I sure did for the old way things downloaded.

mshiels commented 2 years ago

Oh I see it's now embeded in video - will require extraction. Sorry for any confusion!! We may want to do some login code for CBC, but I know my old login code I tried to get working was for clearleap, so that won't work now.

makew0rld commented 2 years ago

@mshiels yep, the subtitles are embedded in the video. The video comes with multiple subtitle tracks, but it seems only the first one has any content I think.

As for login, it isn't needed anymore. CBC logged-in content can be accessed without a username and password.

mshiels commented 2 years ago

I am really glad I checked out yt-dlp since I was about to try and do the exact same thing you just did, which means now I can concentrate on trying to get my acorn.tv one working again - something is wrong and it's been way too long since I worked on it to tell what's up. Thanks for your work on this for sure!!!!

yt-dlp / yt-dlp