[Feature Idea] overlay generated voices on original track

[lightlike]

A lot of german media does translation in an interesting way. The original audio gets reduced in volume and the translated audio gets overlayed above that.

I really like this method and it could save a lot of API calls for the TTS services.

This method would even keep some of the backgroud sounds of the original audio to at least some extend.

Example

You have an audio section and translated audio file in the video that is looks like this:

O: Original T: Translation

O: |----------------------------|
T: |--------------|

To merge these Tracks without using the twopass method could look something like this:

O: |---v------------------^----|
T:       |--------------|

v: turn down the volume ^: turn the audio back up

Possible problems

One Problem could be when the translated audio is longer than the original track. This would still need a second pass to the TTS API but it would reduce the amound of calls to the API saving money.

Config

Some of the configuration options could be something like this:

• whether the generated audio is centered on the original audio
• what offset the start of this audio has from the start of the section (most of the time it has a short delay when I have seen this method)
• behaviour for when the translation is longer than the original section
• volume of the turned down audio

Existing examples

This was done for stuff like Top Gear, Pawn Stars and many documentaries. At least those come to memory for me. One example is something like this: https://www.youtube.com/watch?v=121t4E3EM48 (first thing I found while searching)

[divadsn]

@lightlike FYI the link you provided is only available from Germany, had to turn on my VPN in order to see

[lightlike]

Hmmm. Thats a bit annoying. I'll try to find something better but it's kind of hard to find something translated on the Internet as most of the videos are only using the original audio. And I suspect that any video i'll find on youtube has the same problem.

[ThioJoe]

Hm I know the type of dubbing you're talking about. Might be possible to implement as an optional feature in the future.