search

baxtree / subaligner

Automatically synchronize and translate subtitles, or create new ones by transcribing, using pre-trained DNNs, Forced Alignments and Transformers. https://subaligner.readthedocs.io/
https://hub.docker.com/r/baxtree/subaligner
MIT License
437 stars 18 forks source link

Support for long span subtitles? #49

Closed jerrylususu closed 3 years ago

jerrylususu commented 3 years ago

I am working on a project which involves long span subtitles (20s for a line of subtitle, with multiple words). I am wondering whether it is possible to use subaligner for such scenario. If that's not possible, if I split the words manually, like every 5 words for a new line of subtitle, will it work?

baxtree commented 3 years ago

Hi, @jerrylususu . I tried with a re-fragmented subtitle file spanning 2 hours and 19 minutes (original cues were regrouped into new cues each of which spans at least 20 seconds) and that still works. It is worth noting subaligner does not alter the numbers of words or lines so 20-second subtitle could end up with being displayed as a large piece of text over your video frames (of course, this varies for different languages). To split the words automatically, you probably need ASR in order to detect their temporal boundaries.

jerrylususu commented 3 years ago

Thanks for the reply! This sounds amazing. Currently I am thinking using some simple heuristics (such as split by punctuation) to limit the amount of words shown, and use subaligner to fix the time mismatch. I will close the issue for now, and make some more reply when my results are available.