Comedy Central Extractor Fails to find video mgid for 12/1 Extended At Midnight Episode

jdunn0 commented 7 years ago

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.12.01. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2016.12.01

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

$ youtube-dl -f "best[height=720]"  --autonumber-size 1 -o "DL%(autonumber)s.mp4" http://www.cc.com/full-episodes/rfft2t/-midnight-with-chris-hardwick-extended---thursday--december-1--2016---uncensored-season-4-ep-4031 -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-f', u'best[height=720]', u'--autonumber-size', u'1', u'-o', u'DL%(autonumber)s.mp4', u'http://www.cc.com/full-episodes/rfft2t/-midnight-with-chris-hardwick-extended---thursday--december-1--2016---uncensored-season-4-ep-4031', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.11.18
[debug] Python version 2.7.12 - Linux-4.4.21-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700K_CPU_@_4.00GHz-with-gentoo-2.2
[debug] exe versions: ffmpeg 2.8.6, ffprobe 2.8.6, rtmpdump 2.4
[debug] Proxy map: {}
[ComedyCentral] -midnight-with-chris-hardwick-extended---thursday--december-1--2016---uncensored-season-4-ep-4031: Downloading webpage
ERROR: Unable to extract mgid; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/mtv.py", line 208, in _extract_mgid
    og_url = self._og_search_video_url(webpage)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/common.py", line 778, in _og_search_video_url
    return self._html_search_regex(regexes, html, name, **kargs)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/common.py", line 669, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/common.py", line 660, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract video url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 694, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/common.py", line 357, in extract
    return self._real_extract(url)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/mtv.py", line 230, in _real_extract
    mgid = self._extract_mgid(webpage)
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/mtv.py", line 224, in _extract_mgid
    r'embed/(mgid:.+?)["\'&?/]', sm4_embed, 'mgid')
  File "/usr/lib64/python2.7/site-packages/youtube_dl/extractor/common.py", line 660, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract mgid; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Video: http://www.cc.com/full-episodes/rfft2t/-midnight-with-chris-hardwick-extended---thursday--december-1--2016---uncensored-season-4-ep-4031

Description of your issue, suggested solution and other information

The Comedy Central extractor fails to find the mgid for the 12/01 Extended At Midnight Episode above. For whatever reason this video page seems to not have the mgid where youtube-dl expects it to be.

Using the network tab in Firefox when loading the page, I was able to locate a json feed with the mgid of the video (mgid:arc:episode:comedycentral.com:e58e0314-8219-11e6-bfd4-0026b9414f30).

I wanted to see if youtube-dl could download the video with the right mgid so I edited the file extractor/mtv.py and replaced the _extract_mgid function with one that just returned the mgid I found. After doing this, youtube-dl was able to download the episode without any problems.

So it seems the code for finding the mgid needs to be updated to handle this video.

dstftw commented 7 years ago

You are using an outdated version of youtube-dl. Read this FAQ entry if you have troubles updating.

jdunn0 commented 7 years ago

Huh, I thought I had installed 2016.12.01 when I had this issue. I just typed emerge -av youtube-dl and it showed that I apparently had not updated it. I hit enter to install the new version and then youtube-dl had no problem downloading this video. I remember specifically being surprised that the new version got in the gentoo repo so quickly but it seems I forgot to actually install the new version that was available.

ytdl-org / youtube-dl