search

pannal / Sub-Zero.bundle

Subtitles for Plex, as good you would expect them to be.
Other
1.76k stars 120 forks source link

Subs out of sync #116

Closed joakim93 closed 8 years ago

joakim93 commented 8 years ago

Hi this plugin has worked quite good but every now and then i get out of sync subtitles. At the beginning the come maybe a second before then 5-6 min in they come perfect then they start to come after. In the end you can't even watch it anymore.. It is exactly the same problem with every one of my languages. English, Swedish and Finnish.. And it doesn't help to force refresh.. But when i get a different quality it works, a different for the episode. The problem is always when i have a bluray version.. HDTV and Web-dl works. I saved the logs after i did a force refresh on The flash season 1 episode 3.. The file is named "The Flash (2014) - S01E02 - Fastest Man Alive 1080p BluRay rovers" In a folder structure: The Flash (2014) - Season 1. And i always have the last version of Sub-Zero and plex i have a script that updates them once a night. Here are the logs.. https://gist.github.com/joakim93/4e3f269976828dcd9229

Thanks

pannal commented 8 years ago

That's a bad match. Next SZ version will have framerate detection at least for OpenSubtitles, which should fix most of those problems.

joakim93 commented 8 years ago

okay :) do you think it's a bad match on every one that I have?

pannal commented 8 years ago

I don't know, I don't see any downloads for The Flash in your log. All it does is find the existing subs. Could you delete all subs for one episode, refresh it, and post the log?

joakim93 commented 8 years ago

here it is : https://gist.github.com/joakim93/0ee45233523bb94ba7ca

pannal commented 8 years ago

That log is cut off, please post the .1 log file, it downloads an OpenSubtitles subtitle with score 141 but I don't see the score computation in that log.

joakim93 commented 8 years ago

Do you see from this? https://gist.githubusercontent.com/joakim93/f2b0c988b69b6e5bf14c/raw/6104b3806daff6b2e34ed6e6f6be7e55a94ab3c1/com.plexapp.agents.subzero.log.1

pannal commented 8 years ago

Now the one without the .1 is gone again, can you repost it?

joakim93 commented 8 years ago

https://gist.github.com/joakim93/5b0562f33d128896e948

pannal commented 8 years ago

I think I got it. It should download this one:

<PodnapisiSubtitle 'http://www.podnapisi.net/subtitles/sv-the-flash-2014-S01E02/Ceg9' [sv]>
2016-01-29 00:17:48,644 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: type=episode (None) (confidence=1.00)
2016-01-29 00:17:48,645 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: container=mkv (None) (confidence=1.00)
2016-01-29 00:17:48,656 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Found match tree:
00000000000000000000000000000000000000000000000000 111
00000000000000000000000000000000000000000000000000 000
00000000000000000000000000001111112222222222222222 000
0000000000000000000000111112      0111122222222222
0000000000000001111112                 00000000111
000000000011112                        01111112
______________.______._____.______.____-______.___.___
TTTTTTTTTTyyyy EEEEEE sssss ffffff vvvv rrrrrr lll ccc
The.Flash.2014.S01E02.1080p.BluRay.x264-ROVERS.swe.mkv
2016-01-29 00:17:48,656 (808b80c00) :  DEBUG (logger:28) - guessit.guess: Updating matching property 'container' with confidence 1.00
2016-01-29 00:17:48,657 (808b80c00) :  DEBUG (logger:28) - guessit.matchtree: Final result: {
    [0.60] "episodeNumber": 2, 
    [0.60] "videoCodec": "h264", 
    [1.00] "container": "mkv", 
    [0.54] "language": [
        "sv"
    ], 
    [0.60] "format": "BluRay", 
    [0.42] "series": "The Flash", 
    [0.60] "season": 1, 
    [0.60] "releaseGroup": "ROVERS", 
    [0.60] "screenSize": "1080p", 
    [0.60] "year": 2014, 
    [1.00] "type": "episode"
}
2016-01-29 00:17:48,658 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: type=episode (None) (confidence=1.00)
2016-01-29 00:17:48,659 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: container=mkv (None) (confidence=1.00)
2016-01-29 00:17:48,669 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Found match tree:
000000000000000000000000000000000000000000 111
000000000000000000000000000000000000000000 000
000000000000000000000000111111222222222222 000
000000000000000000011112      011112222222
0000000000000001112                0111111
000000000011112
______________.___.____.______.____-______.___
TTTTTTTTTTyyyy SSS ssss ffffff vvvv rrrrrr ccc
The.Flash.2014.S01.720p.BluRay.x264-DEMAND.mkv
2016-01-29 00:17:48,669 (808b80c00) :  DEBUG (logger:28) - guessit.guess: Updating matching property 'container' with confidence 1.00
2016-01-29 00:17:48,670 (808b80c00) :  DEBUG (logger:28) - guessit.matchtree: Final result: {
    [0.60] "videoCodec": "h264", 
    [1.00] "container": "mkv", 
    [0.60] "format": "BluRay", 
    [0.24] "series": "The Flash", 
    [0.36] "season": 1, 
    [0.60] "releaseGroup": "DEMAND", 
    [0.60] "screenSize": "720p", 
    [0.60] "year": 2014, 
    [1.00] "type": "episode"
}
2016-01-29 00:17:48,671 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: type=episode (None) (confidence=1.00)
2016-01-29 00:17:48,671 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: container=mkv (None) (confidence=1.00)
2016-01-29 00:17:48,681 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Found match tree:
0000000000000000000000000000000000000000000 111
0000000000000000000000000000000000000000000 000
0000000000000000000000000111111222222222222 000
0000000000000000000111112      011112222222
0000000000000001112                 0111111
000000000011112
______________.___._____.______.____-______.___
TTTTTTTTTTyyyy SSS sssss ffffff vvvv rrrrrr ccc
The.Flash.2014.S01.1080p.BluRay.x264-ROVERS.mkv
2016-01-29 00:17:48,682 (808b80c00) :  DEBUG (logger:28) - guessit.guess: Updating matching property 'container' with confidence 1.00
2016-01-29 00:17:48,682 (808b80c00) :  DEBUG (logger:28) - guessit.matchtree: Final result: {
    [0.60] "videoCodec": "h264", 
    [1.00] "container": "mkv", 
    [0.60] "format": "BluRay", 
    [0.24] "series": "The Flash", 
    [0.36] "season": 1, 
    [0.60] "releaseGroup": "ROVERS", 
    [0.60] "screenSize": "1080p", 
    [0.60] "year": 2014, 
    [1.00] "type": "episode"
}
2016-01-29 00:17:48,683 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: type=episode (None) (confidence=1.00)
2016-01-29 00:17:48,684 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: container=mkv (None) (confidence=1.00)
2016-01-29 00:17:48,693 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Found match tree:
000000000000000000000000000000000000 111
000000000000000000000000000000000000 000
000000000000000000011111222222222222 000
0000000000000001112     011112222222
000000000011112              0111111
______________.___._____.____-______.___
TTTTTTTTTTyyyy SSS fffff vvvv rrrrrr ccc
The.Flash.2014.S01.BDRip.x264-DEMAND.mkv
2016-01-29 00:17:48,693 (808b80c00) :  DEBUG (logger:28) - guessit.guess: Updating matching property 'container' with confidence 1.00
2016-01-29 00:17:48,693 (808b80c00) :  DEBUG (logger:28) - guessit.matchtree: Final result: {
    [0.60] "videoCodec": "h264", 
    [1.00] "container": "mkv", 
    [0.60] "format": "BluRay", 
    [0.24] "series": "The Flash", 
    [0.36] "season": 1, 
    [0.60] "releaseGroup": "DEMAND", 
    [0.60] "year": 2014, 
    [1.00] "type": "episode"
}
2016-01-29 00:17:48,694 (808b80c00) :  INFO (logger:30) - subliminal_patch.patch_subtitle: Computing score for matches set([u'episode', u'format', u'series', u'year', u'release_group', u'season', u'video_codec', u'resolution', u'hearing_impaired']) and <Episode [u'The Flash', 2014, 1x2]>
2016-01-29 00:17:48,694 (808b80c00) :  DEBUG (logger:28) - subliminal_patch.patch_subtitle: Final matches: set([u'episode', u'format', u'series', u'season', u'hearing_impaired', u'year', u'video_codec', u'resolution', u'release_group'])
2016-01-29 00:17:48,694 (808b80c00) :  INFO (logger:30) - subliminal_patch.patch_subtitle: Computed score 136

But that scores only 136, even though it matches crucial data.

Instead it downloads this one:

2016-01-29 00:17:47,318 (808b80c00) :  DEBUG (logger:28) - subliminal_patch.patch_provider_pool: Starting score computation for <PatchedOpenSubtitlesSubtitle 'http://www.opensubtitles.org/en/subtitles/6410946/sid-8licg3r367k12u909kqmvto647/the-flash-fastest-man-alive-sv' [sv]>
2016-01-29 00:17:47,319 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: type=episode (None) (confidence=1.00)
2016-01-29 00:17:47,320 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Property found: container=mkv (None) (confidence=1.00)
2016-01-29 00:17:47,331 (808b80c00) :  DEBUG (logger:28) - guessit.matcher: Found match tree:
0000000000000000000000000000000000000000000000 111
0000000000000000000000000000000000000000000000 000
0000000000000000000000000001111222222222222222 000
000000000000000000000011112    011112222222222
0000000000000001111112              0111111111
000000000011112
______________.______.____.____.____-_________.___
TTTTTTTTTTyyyy EEEEEE ssss ffff vvvv rrrrrrrrr ccc
The.Flash.2014.S01E02.720p.HDTV.X264-DIMENSION.mkv
2016-01-29 00:17:47,332 (808b80c00) :  DEBUG (logger:28) - guessit.guess: Updating matching property 'container' with confidence 1.00
2016-01-29 00:17:47,332 (808b80c00) :  DEBUG (logger:28) - guessit.matchtree: Final result: {
    [0.60] "episodeNumber": 2, 
    [0.60] "videoCodec": "h264", 
    [1.00] "container": "mkv", 
    [0.60] "format": "HDTV", 
    [0.42] "series": "The Flash", 
    [0.60] "season": 1, 
    [0.60] "releaseGroup": "DIMENSION", 
    [0.60] "screenSize": "720p", 
    [0.60] "year": 2014, 
    [1.00] "type": "episode"
}
2016-01-29 00:17:47,333 (808b80c00) :  INFO (logger:30) - subliminal_patch.patch_subtitle: Computing score for matches set([u'episode', u'title', u'series', u'year', u'season', u'video_codec', u'hearing_impaired']) and <Episode [u'The Flash', 2014, 1x2]>
2016-01-29 00:17:47,333 (808b80c00) :  DEBUG (logger:28) - subliminal_patch.patch_subtitle: Final matches: set([u'episode', u'hearing_impaired', u'video_codec', u'title', u'series', u'year', u'season'])
2016-01-29 00:17:47,333 (808b80c00) :  INFO (logger:30) - subliminal_patch.patch_subtitle: Computed score 137

The last one, which is the wrong choice, scores higher, because the title attribute gets scored too high. I'm not even sure, why the title attribute even matched.

Still, I guess I will be lowering the score for "title" to 22 instead of 44 for episodes, to circumvent that. As the title attribute isn't very important for an episode match, this may even get taken out of the equasion totally.

@Diaoul What do you think?

joakim93 commented 8 years ago

Okay thank you..

I'll try this when you push a fix and then we'll see if my other sync problems goes away :)

Diaoul commented 8 years ago

title is equivalent to matching season + episode: https://github.com/Diaoul/subliminal/blob/develop/subliminal/score.py#L60 Here is the part where the deduplication is done in subliminal to keep score consistent: https://github.com/Diaoul/subliminal/blob/develop/subliminal/subtitle.py#L204-L205 You removed it in your version: https://github.com/pannal/Sub-Zero.bundle/blob/master/Contents/Libraries/Shared/subliminal_patch/patch_subtitle.py#L49-L53

pannal commented 8 years ago

Hmm I see. I must have misunderstood that.

I've removed that equasion before, because I had users where the episode title matched, but season and episode were wrong (which may have been unrelated to that equasion). Doesn't that mean that a subtitle with matching episode and season is worth the same as a subtitle with matching episode title but a completely wrong season/episode number? That sounds wrong to me.

This flat-out adds 44 if the episode has a title and it matches the detected title, doesn't it? https://github.com/pannal/Sub-Zero.bundle/blob/master/Contents/Libraries/Shared/subliminal/providers/opensubtitles.py#L67

Or am I again misunderstanding the scoring?

Diaoul commented 8 years ago

That's correct, they're worth the same with the default implementation. You changed it so title is a boost now. I'd say both implementation are wrong.

Currently there's no indication if it's a mismatch or no match. There is some information loss in the compute_matches logic as we return what matches but not what is there but does not match. For example, if season is 2 in the subtitle and 1 in the video it's worse IMO than if it's unknown in the subtitle and 1 in the video. So subliminal should handle that.

I think I'll change the way matches are returned to return a tuple (good_matches, bad_matches) even if that's not the main issue. I'll also make the compute_score function to use customizable in download_best_subtitles and it'll accept the subtitle and the video as positional arguments and will be responsible for returning the key to sort the subtitles. This should allow more flexibility to implement a custom scoring algorithm.

pannal commented 8 years ago

That sounds good. In the meantime I think I will remove the title = season + episode equasion in the scoring as that should temporarily fix the problem. Or perhaps the scoring for a title of an episode altogether.

Btw, in my first log quote, why didn't it include "title" in the final matches also? set([u'episode', u'format', u'series', u'season', u'hearing_impaired', u'year', u'video_codec', u'resolution', u'release_group']) - it has season and episode in it.

pannal commented 8 years ago

Good matches and bad matches would also mean my OpenSubtitles FPS fix could be implemented properly: https://github.com/pannal/Sub-Zero.bundle/blob/develop/Contents/Libraries/Shared/subliminal_patch/patch_providers/opensubtitles.py#L25

pannal commented 8 years ago

@joakim93 please try https://github.com/pannal/Sub-Zero.bundle/archive/title_match.zip and report back.

joakim93 commented 8 years ago

how do i test it? i tried to replace subzero.bundle with this but it doesn't show up in plex. Then i updated the whole git and tried to do a git chekcout title_match but that didn't work either. But i tried develop yesterday after you did the change from 44 points to 22 and that worked because i got sub that were in sync. I tried only the one flash episode and that worked

pannal commented 8 years ago

OK. You should be able to do just the same as you did when you used the develop branch. Delete Sub-Zero.bundle and unpack the new one. Rename the unpacked folder to Sub-Zero.bundle and restart your PMS.

joakim93 commented 8 years ago

Yeah that did the trick. English subs works and Swedish subs work but the finnis subs are still out of sync. I'll post logs when i come home and have checked that there indeed are finnish subs out there that are in sync. https://gist.githubusercontent.com/joakim93/c86a2674643b3f3fe128/raw/3710fa85fdc3f54f1d8ea5d18768e501937b29b0/com.plexapp.agents.subzero.log

pannal commented 8 years ago

Yeah it detects 0 fps for the correct subtitle and ignores it. Fixing it in the develop branch.

Update: This actually is unrelated. See below.

pannal commented 8 years ago

Pushed the change to the develop branch. Please retry. Keep in mind though, that you've hit your daily limit on OpenSubtitles:

2016-01-30 20:37:43,302 (808b7e400) :  ERROR (logger:38) - subliminal_patch.patch_provider_pool: Unexpected error in provider u'opensubtitles', Traceback: Traceback (most recent call last):
  File "/var/db/plexdata/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal_patch/patch_provider_pool.py", line 214, in download_subtitle
    self[subtitle.provider_name].download_subtitle(subtitle)
  File "/var/db/plexdata/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal/providers/opensubtitles.py", line 185, in download_subtitle
    response = checked(self.server.DownloadSubtitles(self.token, [str(subtitle.subtitle_id)]))
  File "/var/db/plexdata/Plex Media Server/Plug-ins/Sub-Zero.bundle/Contents/Libraries/Shared/subliminal/providers/opensubtitles.py", line 243, in checked
    raise DownloadLimitReached
DownloadLimitReached

Update: Actually I don't see any perfect match for your The Flash S01E01 file on OpenSubtitles for Finnish btw.

pannal commented 8 years ago

Fixes implemented in newest release.