[Funimation] error getting file information and downloading it

CZeil commented 2 years ago

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2021.12.27. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region

US

Description

I was trying to see if the error was mostly mine but it seems to be a coding error, I think.

link: https://www.funimation.com/v/the-daily-life-of-the-immortal-king/wang-ling-wants-to-lead-a-quiet-life

Verbose log

C:\youtube-dl>yt-dlp -F --cookies cookies-funimation-com.txt https://www.funimation.com/v/the-daily-life-of-the-immortal-king/wang-ling-wants-to-lead-a-quiet-life --list-subs -Uv
[debug] Command-line config: ['-F', '--cookies', 'cookies-funimation-com.txt', 'https://www.funimation.com/v/the-daily-life-of-the-immortal-king/wang-ling-wants-to-lead-a-quiet-life', '--list-subs', '-Uv']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2021.12.27 [6223f67a8] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: f76ca2dd5
[debug] Python version 3.9.7 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg N-105035-g617452ce2c-20211227 (setts), ffprobe N-105035-g617452ce2c-20211227
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2021.12.27, Current version: 2021.12.27
yt-dlp is up to date (2021.12.27)
[debug] [funimation:page] Extracting URL: https://www.funimation.com/v/the-daily-life-of-the-immortal-king/wang-ling-wants-to-lead-a-quiet-life
[funimation:page] the-daily-life-of-the-immortal-king_wang-ling-wants-to-lead-a-quiet-life: Downloading JSON metadata
[debug] [Funimation] Extracting URL: https://www.funimation.com/player/3905567
[Funimation] 3905567: Downloading player webpage for 3905567
ERROR: [Funimation] 3905567: Unable to extract show data; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "C:\Program Files\youtube-dl\yt-dlp\yt_dlp\extractor\common.py", line 609, in extract
    ie_result = self._real_extract(url)
  File "C:\Program Files\youtube-dl\yt-dlp\yt_dlp\extractor\funimation.py", line 193, in _real_extract
    episode, season, show = self._get_episode(webpage, experience_id=int(initial_experience_id))
  File "C:\Program Files\youtube-dl\yt-dlp\yt_dlp\extractor\funimation.py", line 171, in _get_episode
    self._search_regex(
  File "C:\Program Files\youtube-dl\yt-dlp\yt_dlp\extractor\common.py", line 1183, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)

Jules-A commented 2 years ago

This show doesn't have a player page. It looks like Funimation want to move away from using the player page and start relying solely on apis.

        ''' Extract the episode, season and show objects given either episode/experience id '''
        show = self._parse_json(
            self._search_regex(
                r'show\s*=\s*({.+?})\s*;', webpage, 'show data', fatal=fatal),
            experience_id, transform_source=js_to_json, fatal=fatal) or []
        for season in show.get('seasons', []):
            for episode in season.get('episodes', []):
                if episode_id is not None:
                    if str(episode.get('episodePk')) == episode_id:
                        return episode, season, show
                    continue
                for _, _, f in self._get_experiences(episode):
                    if f.get('experienceId') == experience_id:
                        return episode, season, show
        if fatal:
            raise ExtractorError('Unable to find episode information')
        else:
            self.report_warning('Unable to find episode information')
        return {}, {}, {}

Pretty sure this can all be done better using the apis anyway.

CZeil commented 2 years ago

This show doesn't have a player page. It looks like Funimation want to move away from using the player page and start relying solely on apis.

        ''' Extract the episode, season and show objects given either episode/experience id '''
        show = self._parse_json(
            self._search_regex(
                r'show\s*=\s*({.+?})\s*;', webpage, 'show data', fatal=fatal),
            experience_id, transform_source=js_to_json, fatal=fatal) or []
        for season in show.get('seasons', []):
            for episode in season.get('episodes', []):
                if episode_id is not None:
                    if str(episode.get('episodePk')) == episode_id:
                        return episode, season, show
                    continue
                for _, _, f in self._get_experiences(episode):
                    if f.get('experienceId') == experience_id:
                        return episode, season, show
        if fatal:
            raise ExtractorError('Unable to find episode information')
        else:
            self.report_warning('Unable to find episode information')
        return {}, {}, {}

Pretty sure this can all be done better using the apis anyway.

But I found it strange because the chapters of the second season are the problem, the first season I can download normally.

nyxtyr commented 2 years ago

Can we fix this error on our end or do we need to wait for an update?

CZeil commented 2 years ago

Can we fix this error on our end or do we need to wait for an update?

I imagine that if you have knowledge of python and know how to solve the problem there is no problem.

Jules-A commented 2 years ago

You can get all the information needed using the api: https://prod-api-funimationnow.dadcdigital.com/api/source/catalog/episode/the-daily-life-of-the-immortal-king/wang-ling-wants-to-lead-a-quiet-life/

But I'm no good doing this stuff, I might give it a shot I guess but even if I get it working it would only be a workaround, someone experienced would need to do it.

EDIT: Nvm... it's too much for me to do :(

hama3254 commented 2 years ago

i can explain how funimation works right now but i can not write anything in python so i can not fix the issue.

i don't know where Jules got that API url from but the desktop api url for a single episode is https://d33et77evd9bgg.cloudfront.net/data/v1/episodes/[Title].json

In the case of the video above the [Title] would be replaced with the episode title that is inside the url wang-ling-wants-to-lead-a-quiet-life

this gives us a json file with a bunch of information of which we directly only need the id which in this case would be TDI0016, there are other usefull things like the title of the episode with a more readable way and we also get the season/episode numbers and the Anime title. this gives us everything we need to properly save the file exept the video/subtitle files we want.

To get that url there is an second api that is either with it simple "anonymous usecase" https://playback.prd.funimationsvc.com/v1/play/anonymous/[id]?deviceType=web or the advanced version https://playback.prd.funimationsvc.com/v1/play/[id]?deviceType=web which requires an Authorization http header with an Token which is just the value of the src_token cookie grafik

The anonymous works fine for all free episodes as long there are not locked behind the mature content filter.

The second API repsond with an other json that contains all available streams and subtitles, which can depends on the location and account type.

yt-dlp / yt-dlp