search

ThioJoe / Auto-Synced-Translated-Dubs

Automatically translates the text of a video based on a subtitle file, and then uses AI voice services to create a new dubbed & translated audio track where the speech is synced using the subtitle's timings.
GNU General Public License v3.0
1.56k stars 156 forks source link

The dubbing is not respecting the pause that exists in the subtitles. #51

Open Fiaspera opened 1 year ago

Fiaspera commented 1 year ago

I'm a Brazilian content creator and I'm using Google, I couldn't get Azure to work.

I generated 3 voiceovers, English, Spanish and Chinese to test. The quality is pretty cool, but when checking the editor's timeline, in some moments the subtitle pause time is respected, but in others, it is completely ignored, and it is very out of sync with what is being shown in the video . There are several moments in the video where there is no narration and the dubbing did not respect these times.

In Chinese for example, he respected the pauses 3 times in the entire video, which was totally out of sync. image

ThioJoe commented 1 year ago

Hmm that's odd, can you provide the original subtitles srt file so I can check it out? Also so I understand, you mean there are time gaps between text lines in the original subtitles that you want to maintain, but those gaps disappear?

I'm wondering if maybe it's combining lines when it shouldn't be. You may be able to try setting combine_subtitles_max_chars in config.ini to 0, which would disable them being combined altogether.

Fiaspera commented 1 year ago

It worked with your tip!

Follow the caption in case you're still curious

subtitles.txt