ScienceChannel.com shows as unsupported after site changes

Sipherdrakon commented 3 years ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.06.06
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log


[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-o', '%(series)s - 0%(season_number)sx0%(episode_number)s - %(title)s [WEBDL-%(height)s].%(ext)s', '-f', 'best', '-ic', '--newline', '--cookies', 'cookies.txt', '--console-title', '--all-subs', '--embed-subs', '-a', 'ytdlready_sci.txt', '--verbose']
[debug] Batch file urls: ['https://www.sciencechannel.com/video/impossible-engineering-science/inside-the-super-stadium', 'https://www.sciencechannel.com/video/mysteries-of-the-abandoned-science/ghosts-of-mount-fortune', 'https://www.sciencechannel.com/video/strangest-things-science-atve-us/nazi-mystery-machine', 'https://www.sciencechannel.com/video/unearthed-science/legend-of-xanadu']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.9.4 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev, ffprobe 4.4-full_build-www.gyan.dev
[debug] Proxy map: {}
[generic] inside-the-super-stadium: Requesting header
WARNING: Falling back on generic information extractor.
[generic] inside-the-super-stadium: Downloading webpage
[generic] inside-the-super-stadium: Extracting information
ERROR: Unsupported URL: https://www.sciencechannel.com/video/impossible-engineering-science/inside-the-super-stadium
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.sciencechannel.com/video/impossible-engineering-science/inside-the-super-stadium

[generic] ghosts-of-mount-fortune: Requesting header
WARNING: Falling back on generic information extractor.
[generic] ghosts-of-mount-fortune: Downloading webpage
[generic] ghosts-of-mount-fortune: Extracting information
ERROR: Unsupported URL: https://www.sciencechannel.com/video/mysteries-of-the-abandoned-science/ghosts-of-mount-fortune
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.sciencechannel.com/video/mysteries-of-the-abandoned-science/ghosts-of-mount-fortune

[generic] nazi-mystery-machine: Requesting header
WARNING: Falling back on generic information extractor.
[generic] nazi-mystery-machine: Downloading webpage
[generic] nazi-mystery-machine: Extracting information
ERROR: Unsupported URL: https://www.sciencechannel.com/video/strangest-things-science-atve-us/nazi-mystery-machine
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.sciencechannel.com/video/strangest-things-science-atve-us/nazi-mystery-machine

[generic] legend-of-xanadu: Requesting header
WARNING: Falling back on generic information extractor.
[generic] legend-of-xanadu: Downloading webpage
[generic] legend-of-xanadu: Extracting information
ERROR: Unsupported URL: https://www.sciencechannel.com/video/unearthed-science/legend-of-xanadu
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "C:\Program Files\Python39\lib\site-packages\youtube_dl-2021.6.6-py3.9.egg\youtube_dl\extractor\generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.sciencechannel.com/video/unearthed-science/legend-of-xanadu```

## Description

<!--
Provide an explanation of your issue in an arbitrary form. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->

    _VALID_URL = r'''(?x)https?://
        (?P<site>
            go\.discovery|
            www\.
                (?:
                    investigationdiscovery|
                    discoverylife|
                    animalplanet|
                    ahctv|
                    destinationamerica|
                    sciencechannel|
                    tlc
                )|
            watch\.
                (?:
                    hgtv|
                    foodnetwork|
                    travelchannel|
                    diynetwork|
                    cookingchanneltv|
                    motortrend
                )
        )\.com/(?:video|tv-shows)/(?P<show_slug>[^/]+)/(?:videos/|full-episodes/|)(?P<id>[^./?#]+)'''

I changed the discovery.py to allow for the inclusion of the new site structure but only got a json error for my efforts. :-(

Sipherdrakon commented 3 years ago

json error as follows:

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-o', '%(series)s - 0%(season_number)sx0%(episode_number)s - %(title)s [WEBDL-%(height)s].%(ext)s', '-f', 'best', '-ic', '--newline', '--cookies', 'cookies.txt', '--console-title', '--all-subs', '--embed-subs', '-a', 'ytdlready_sci.txt', '--verbose'] [debug] Batch file urls: ['https://www.sciencechannel.com/video/impossible-engineering-science/inside-the-super-stadium', 'https://www.sciencechannel.com/video/mysteries-of-the-abandoned-science/ghosts-of-mount-fortune', 'https://www.sciencechannel.com/video/strangest-things-science-atve-us/nazi-mystery-machine', 'https://www.sciencechannel.com/video/unearthed-science/legend-of-xanadu'] [debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252 [debug] youtube-dl version 2021.06.06 [debug] Git HEAD: 199b483cb [debug] Python version 3.9.4 (CPython) - Windows-10-10.0.18362-SP0 [debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev, ffprobe 4.4-full_build-www.gyan.dev [debug] Proxy map: {} [Discovery] inside-the-super-stadium: Downloading token JSON metadata ERROR: inside-the-super-stadium: Failed to parse JSON (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "D:\youtube-dl\youtube_dl\extractor\common.py", line 906, in _parse_json return json.loads(json_string) File "C:\Program Files\Python39\lib\json__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Program Files\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Program Files\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last): File "D:\youtube-dl\youtube_dl\extractor\common.py", line 906, in _parse_json return json.loads(json_string) File "C:\Program Files\Python39\lib\json__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Program Files\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Program Files\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\youtube-dl\youtube_dl\YoutubeDL.py", line 815, in wrapper return func(self, *args, **kwargs) File "D:\youtube-dl\youtube_dl\YoutubeDL.py", line 836, in __extract_info ie_result = ie.extract(url) File "D:\youtube-dl\youtube_dl\extractor\common.py", line 534, in extract ie_result = self._real_extract(url) File "D:\youtube-dl\youtube_dl\extractor\discovery.py", line 79, in _real_extract access_token = self._download_json( File "D:\youtube-dl\youtube_dl\extractor\common.py", line 895, in _download_json res = self._download_json_handle( File "D:\youtube-dl\youtube_dl\extractor\common.py", line 881, in _download_json_handle return self._parse_json( File "D:\youtube-dl\youtube_dl\extractor\common.py", line 910, in _parse_json raise ExtractorError(errmsg, cause=ve) youtube_dl.utils.ExtractorError: inside-the-super-stadium: Failed to parse JSON (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Sipherdrakon commented 3 years ago

Hmm, maybe it will work better with adjusting dplay.py

dirkf commented 3 years ago

Have a look at #29249, where I attached a patch for the European D+.

ytdl-org / youtube-dl

ScienceChannel.com shows as unsupported after site changes #29634

Checklist

Verbose log