Problems with Disney: Fails to get series info (1 of 2)

keybounce commented 7 years ago

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2017.06.05. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2017.06.05

Before submitting an issue make sure you have:

[ x] At least skimmed through README and most notably FAQ and BUGS sections
[ x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

x ] Bug report (encountered problems with youtube-dl)
[ x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Trying to fetch episodes from Disney XD works just fine; passing http://watchdisneyxd.go.com/star-vs-the-forces-of-evil results in identifying the individual episodes, and correctly identifies series, season, and episode.

But trying to fetch from Disney Channel fails.

Giving a specific episode does not retrieve series, season, or episode number

keybounceMBP:Disney michael$ 
keybounceMBP:Disney michael$ ./yt-dl -v --get-filename http://video.disney.com/watch/disneychannel-what-the-ha
ir-54c786cedea9193243001d54
[debug] System config: []
[debug] User config: ['-k', '-o', '%(title)s-%(timestamp)6i.%(ext)s', '-f', '\nbest[ext=mp4][height>431][height<=576]/\nbestvideo[ext=mp4][height=480]+bestaudio[ext=m4a]/\nbest[ext=mp4][height>340][height<=431]/\nbestvideo[ext=mp4][height>360][height<=576]+bestaudio/\nbest[height>340][height<=576]/\nbestvideo[height>360][height<=576]+bestaudio/\nbestvideo[height=360]+bestaudio/\nbest[ext=mp4][height>=280][height<=360]/\nbest[height<=576]/\nworst', '--ap-mso', 'Dish', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--write-sub', '--write-auto-sub', '--sub-lang', 'en,enUS,en-us', '--sub-format', 'ass/srt/best', '--convert-subs', 'ass', '--embed-subs', '--recode-video', 'mp4', '--mark-watched', '--download-archive', 'downloaded-videos.txt']
[debug] Custom config: []
[debug] Command-line args: ['-o', '%(series)s/s%(season_number)02d-e%(episode_number)02d-%(title)s.%(ext)s', '-v', '--get-filename', 'http://video.disney.com/watch/disneychannel-what-the-hair-54c786cedea9193243001d54']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.05
[debug] Python version 3.6.1 - Darwin-13.4.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4, rtmpdump 2.4
[debug] Proxy map: {}
WARNING: enUS subtitles not available for 54c786cedea9193243001d54
WARNING: en-us subtitles not available for 54c786cedea9193243001d54
NA/sNA-eNA-What the Hair!.mp4
keybounceMBP:Disney michael$

keybounce commented 7 years ago

It's even better.

The URL I used there: http://video.disney.com/watch/disneychannel-what-the-hair-54c786cedea9193243001d54

But there's another address for disney channel: http://watchdisneychannel.go.com/tangled-the-series/video/vdka3750184/

That one works.

... So some things have three locations, three different sets of available videos, three different sets of information extractors, ...

keybounceMBP:Disney michael$ ./yt-dl -v http://watchdisneychannel.go.com/tangled-the-series/video/vdka3750184/
01/02-what-the-hair
[debug] System config: []
[debug] User config: ['-k', '-o', '%(title)s-%(timestamp)6i.%(ext)s', '-f', '\nbest[ext=mp4][height>431][height<=576]/\nbestvideo[ext=mp4][height=480]+bestaudio[ext=m4a]/\nbest[ext=mp4][height>340][height<=431]/\nbestvideo[ext=mp4][height>360][height<=576]+bestaudio/\nbest[height>340][height<=576]/\nbestvideo[height>360][height<=576]+bestaudio/\nbestvideo[height=360]+bestaudio/\nbest[ext=mp4][height>=280][height<=360]/\nbest[height<=576]/\nworst', '--ap-mso', 'Dish', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--write-sub', '--write-auto-sub', '--sub-lang', 'en,enUS,en-us', '--sub-format', 'ass/srt/best', '--convert-subs', 'ass', '--embed-subs', '--recode-video', 'mp4', '--mark-watched', '--download-archive', 'downloaded-videos.txt']
[debug] Custom config: []
[debug] Command-line args: ['-o', '%(series)s/s%(season_number)02d-e%(episode_number)02d-%(title)s.%(ext)s', '-v', 'http://watchdisneychannel.go.com/tangled-the-series/video/vdka3750184/01/02-what-the-hair']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.05
[debug] Python version 3.6.1 - Darwin-13.4.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[Go] vdka3750184: Downloading JSON metadata
[Go] VDKA3750184: Retrieving Media Token
[Go] VDKA3750184: Downloading JSON metadata
[Go] VDKA3750184: Downloading m3u8 information
WARNING: en subtitles not available for VDKA3750184
WARNING: enUS subtitles not available for VDKA3750184
[download] S1 E2: What the Hair?! has already been recorded in archive
keybounceMBP:Disney michael$

gkoelln commented 7 years ago

We're dealing with two separate extractors here: Disney and Go.

Making the following changes starting on Line 150 of disney.py may help:

        return {
            'id': video_id,
            'title': title,
            'description': video_data.get('description') or video_data.get('short_desc'),
            'thumbnail': video_data.get('thumb') or video_data.get('thumb_secure'),
            'duration': int_or_none(video_data.get('duration_sec')),
            'upload_date': unified_strdate(video_data.get('publish_date')),
            'formats': formats,
            'subtitles': subtitles,
            'season_number': int_or_none(video_data.get('externals', {})[0]['data'].get('Season')),
            'episode_number': int_or_none(video_data.get('externals', {})[0]['data'].get('Episode')),
            'series': video_data.get('ptitle'),
            'episode': title
        }

keybounce commented 7 years ago

On 2017-06-06, at 11:34 AM, gkoelln notifications@github.com wrote:

We're dealing with two separate extractors here: Disney and Go.

Making the following changes starting on Line 150 of disney.py may help:

    return {
        'id': video_id,
        'title': title,
        'description': video_data.get('description') or video_data.get('short_desc'),
        'thumbnail': video_data.get('thumb') or video_data.get('thumb_secure'),
        'duration': int_or_none(video_data.get('duration_sec')),
        'upload_date': unified_strdate(video_data.get('publish_date')),
        'formats': formats,
        'subtitles': subtitles,
        'season_number': int_or_none(video_data.get('externals', {})[0]['data'].get('Season')),
        'episode_number': int_or_none(video_data.get('externals', {})[0]['data'].get('Episode')),
        'series': video_data.get('ptitle'),
        'episode': title
    }

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Got it, adding 4 new fields. Also, "episode" is not necessarily the same as "title"? ... hmm, something to experiment with.

Entertaining minecraft videos http://YouTube.com/keybounce

keybounce commented 7 years ago

Yep. Works.

keybounceMBP:Disney michael$ ./yt-dl --get-filename http://video.disney.com/watch/disneychannel-what-the-hair-
54c786cedea9193243001d54
WARNING: enUS subtitles not available for 54c786cedea9193243001d54
WARNING: en-us subtitles not available for 54c786cedea9193243001d54
Tangled - The Series/s01-e01-What the Hair!.mp4
keybounceMBP:Disney michael$

Ok, now, how do I send commit 40915c103bb4824bb5f4cf07586d24e4c5ca9964 to you as a pull request? (This is more of a "using git for group development" question).

gkoelln commented 7 years ago

Episode is usually the same as title, but some titles are something like "Tangled - The Series - What the Hair" as opposed to just "What the Hair!?," depending on the extractor. It's a field that I use when necessary for that reason.

saper commented 7 years ago

Since I cannot access videos in question, can you check if a bit more defensive change in #13384 works?

mpenkov commented 6 years ago

@keybounce Just following up on this ticket. Were you able to check if #13384 solves the problem for you? If yes, then we can close this ticket.

ytdl-org / youtube-dl