kanjieater / SubPlz

🫴 Generate accurate subtitles from audio, align existing subs to videos, generate your own Kindle's Immersion Reading like audiobook subs 📖🎧
MIT License
82 stars 9 forks source link

Alignment Problems with Spanish #4

Open spirited1086 opened 1 year ago

spirited1086 commented 1 year ago

I think this has to do more with stable-ts, and not much to do with your program. But I'm posting anyway, as it might help someone else.

What I found was that using this for Spanish, the subtitles were often broken up in an illogical way, e.g., one sentence would be split over a few subtitles, a semi-colon might also split a subtitle, or the last word in a sentence would get its own subtitle line. Once I set 'regroup' to false, this fixed that problem. Now the subtitles themselves are split perfectly, i.e., each sentence gets its own subtitle line, which is exactly what's needed for sentence mining.

However, the alignment is off by some amount every line. It's not consistent so I can't fix it by adjusting the subtitle delay. I've set the model to small, language to es, and regrouping to false. Otherwise, that's all I've changed. Given that it's matching the transcription with actual text. What might cause bad alignment? Is it possible that unrecognized symbols in the text might cause this, e.g., "—", "¿", "«"?

I've checked the text matches and extracted only small portions to make sure there is nothing extra in there other than what's spoken in the audio book.

Am I hoping for too much to expect that I'll get almost consistently good alignment? That is the reason for wanting to use this, after all, to sentence mine efficiently from books.

kanjieater commented 3 months ago

You might want to try again now. Many updates have been and languages besides japanese may work even if I don't official development for them