search

ThioJoe / Auto-Synced-Translated-Dubs

Automatically translates the text of a video based on a subtitle file, and also uses AI voice to dub the video, and synced using the subtitle's timings
GNU General Public License v3.0
1.55k stars 152 forks source link

Restrict maximum and minimum speed factor #6

Closed ShivanshuKantPrasad closed 1 year ago

ShivanshuKantPrasad commented 1 year ago

I just watched your youtube video with Hindi audio track and there were parts of videos where the AI spoke incredibly fast or incredibly slow. I think there needs to be a compromise between millisecond accuracy versus speech speed.

Maybe during the second pass the script can create an array of speed factors and if an element is too high or too low, it can try to average it with it's neighbours. For eg: [ ... 1.05, 1.5, 1.1 ...] would get averaged out to [ ... 1.2, 1.2, 1.2 ...] or something like that.

ThioJoe commented 1 year ago

Yea this is definitely something I'm thinking about how to improve

ThioJoe commented 1 year ago

The latest version should now be a lot better in terms of different parts being wildly different speeds. It will heavily depend on the combine_subtitles_max_chars setting set by the user though. The larger this number, the smoother the speed, but also means more subtitles will be on the screen.

Hindi seems to still be kind of fast because I guess it rephrases it in a way that requires a lot of words, so I'm not sure there is any way around that.