Godzil / Crunchy

Crunchy is capable of downloading anime episodes from the popular CrunchyRoll streaming service.
MIT License
95 stars 19 forks source link

Merged seasons #51

Closed jamtat closed 6 years ago

jamtat commented 7 years ago

Running crunchy http://www.crunchyroll.com/my-teen-romantic-comedy-snafu -v 1 doesn't correctly download season 2.

Adding a debug line to series.js to log the produced page gives this:

{ episodes: 
   [ { address: '/my-teen-romantic-comedy-snafu/episode-1-youth-romantic-comedy-is-wrong-as-i-expected-622107',
       episode: '1',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-2-all-people-surely-have-their-own-worries-622109',
       episode: '2',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-3-sometimes-the-gods-of-rom-coms-do-nice-things-622111',
       episode: '3',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-4-in-other-words-he-doesnt-have-many-friends-622113',
       episode: '4',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-5-once-again-he-turns-back-on-the-path-from-whence-he-came-622115',
       episode: '5',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-6-his-beginning-with-her-finally-ends-622117',
       episode: '6',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-7-anyway-getting-no-rest-even-though-its-summer-break-just-isnt-right-622119',
       episode: '7',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-8-one-day-they-will-learn-the-truth-622121',
       episode: '8',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-9-for-the-third-time-he-turns-back-on-the-path-from-whence-he-came-622123',
       episode: '9',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-10-the-distance-between-them-remains-unchanged-as-the-festival-is-becoming-a-carnival-622125',
       episode: '10',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-11-and-so-the-curtain-on-each-stage-rises-and-the-festival-is-festivaling-its-very-best-622127',
       episode: '11',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-12-thus-his-and-her-and-her-youth-continues-to-be-wrong-622129',
       episode: '12',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-13-and-so-their-festival-will-never-end-627843',
       episode: '13',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-1-nobody-knows-why-they-came-to-the-service-club-678269',
       episode: '1',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-2-his-and-her-love-confessions-will-reach-no-one-678741',
       episode: '2',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-3-quietly-yukinoshita-yukino-makes-a-decision-678743',
       episode: '3',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-4-and-then-yuigahama-yui-makes-a-declaration-678745',
       episode: '4',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-5-the-scent-of-tea-doesnt-fill-that-room-anymore-678747',
       episode: '5',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-6-without-incident-the-congress-dances-but-does-not-progress-678749',
       episode: '6',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-7-yet-that-room-continues-to-play-out-the-endless-days-678751',
       episode: '7',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-8-but-still-hikigaya-hachiman-is-678753',
       episode: '8',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-9-and-yukinoshita-yukino-is-678755',
       episode: '9',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-10-the-thing-that-the-light-in-each-of-their-hands-shines-on-678757',
       episode: '10',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-11-hayama-hayato-always-responds-to-everyones-expectationsc-678759',
       episode: '11',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-12-still-the-thing-he-seeks-is-out-of-reach-and-he-continues-to-mistake-whats-real-678761',
       episode: '12',
       volume: 1 },
     { address: '/my-teen-romantic-comedy-snafu/episode-13-spring-bound-beneath-the-thick-snow-begins-to-sprout-678763',
       episode: '13',
       volume: 1 } ],
  series: 'My Teen Romantic Comedy SNAFU' }

From what I can tell in the page function it attempts to parse a number out of the 'season' header on the page. If it can't find one it just defaults to setting the volume as 1. In an ideal world this should probably fall back to computing the season number based on the number of seasons and the season header's relative ordering in the list of seasons.

jamtat commented 7 years ago

Something like this added to the pages function would probably work as a nice fallback:

var season = $('.list-of-seasons').children().length - $(el).closest('.season').index()
Godzil commented 7 years ago

The problem is it will not work everywhere, I've long time ago played with that and CR pages are not that consistent. It fix for some, and not for others, or make it worse on some.

This is mainly a problem on CR side more than Crunchy unfortunately. That's why Crunchy try to download a file even if the filename already exist by appending a "-1" in the end of the filename, that was the safest way to make sure that if we haven't already downloaded a specific URL it will be.

I will look at your path to see if it is more consistant than it was month ago, but as I'm planning to retrive metadata in a (probably) better way than parsing a webpage, I don't know it if is really worth to spend too much time there.

I will still have a look, but yes I agree there are room for improvment here, but it is mainly CR fault at first to not properly tag seasons

clienthax commented 7 years ago

@Godzil Can you give me a few examples of the troublesome ones, i might have something.. ;)

clienthax commented 7 years ago

Using the crunchyroll api it seems that the season numbers make more sense (sometimes they start at 0/1 but atleast its clear)

get session id
http://api.crunchyroll.com/start_session.0.json?device_id=hacks&device_type=com.crunchyroll.winphone&access_token=z6J2faQjApno1A1

anime list
http://api.crunchyroll.com/list_series.0.json?session_id=%SESSIONID%&media_type=anime

magius bride = 271271 series_id

http://api.crunchyroll.com/list_media.0.json?session_id=%SESSIONID%&media_type=anime&series_id=271271

collection ids - magius
season 1 = 22975
season 2 = 23787

My Teen Romantic Comedy SNAFU 
series id = 253981
http://api.crunchyroll.com/list_media.0.json?session_id=%SESSIONID%&series_id=253981

collection ids
season 1 = 21023
season 2 = 22205
http://api.crunchyroll.com/info.0.json?session_id=%SESSIONID%&collection_id=21023
http://api.crunchyroll.com/info.0.json?session_id=%SESSIONID%&collection_id=22205
clienthax commented 6 years ago

Been working on my own tool and have got the season tagging working nicely soo far. Will be on my git in the next few days~ image

clienthax commented 6 years ago

@jamtat Can you try my downloader with this season?, its geolocked and i don't have access to it atm https://github.com/clienthax/Crunched

Vigilence commented 6 years ago

I've used the linked too and it seems to be working fine. The only issue I see is that the videos downloaded don't have the series title added similar to the picture above, nor the episode name.

An error popped up during initial run SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". Another issue I noticed is if there is an odd numbered episode lets say 12.5 it will name is 13 instead.

Tested on US account.

clienthax commented 6 years ago

@Vigilence Ive uploaded a new release, Can you open an issue on my github for the naming thing as im not able to reproduce it atm :)

Vigilence commented 6 years ago

@clienthax Sure, I'll take a look tonight.

Vigilence commented 6 years ago

@clienthax The issue seems to have been resolved with the newest release as well as that error that popped up during the first run.