devmaxxing / videocr-PaddleOCR

Extract hardcoded subtitles from videos using machine learning
MIT License
142 stars 22 forks source link

Improve subtitle merge algorithm #5

Open devmaxxing opened 2 years ago

devmaxxing commented 2 years ago

The algorithm currently merges 2 subtitles by picking the one with the highest average confidence. Ideally, all characters with high confidence should be used if there is no conflict in their positions e.g.

If merging a (99) b (99) c (99) and a (99) b (99) a (98) c (99), the result should be abac instead of abc

obaidabit commented 1 year ago

Is this the reason why the subtitle appears wrong ( flipped and sometimes in the wrong order)?

devmaxxing commented 1 year ago

@obedabit do you have an example?

obaidabit commented 1 year ago

@oliverfei Yes I do, here is an example, I have the following photo it has two line sentences, and the output is wrong on one line, it should be two lines and the last sentence should be the first.

image

output:

3 00:00:51,416 --> 00:00:55,583 right in front of you, of course you'd be depressed. No, if someone else kissed the person you liked

devmaxxing commented 1 year ago

Ah ok. That's cause the library doesn't account the multiline scenario at the moment which is unrelated to this issue.

obaidabit commented 1 year ago

so should we make an issue for It or you will fix it without one?

devmaxxing commented 1 year ago

I pushed something if you want to try it out

obaidabit commented 1 year ago

It is ok, amazing work

woojh3690 commented 1 year ago

The subtitle merging algorithm used by this library is too bad for subtitles that appear as if they were typed rather than appearing in a single frame.