speech-translator-ext / speech-translator-readme

This repository outlines the procedures and general information for the Speech Translator project.
https://st.kappaflow.dev
35 stars 2 forks source link

[Feature Request] Extra translation and Font support #4

Open 8ego opened 11 months ago

8ego commented 11 months ago

It would be nice to have "secondary" translation, for example, translate English spoken language into Korean and Japanese. Also allowing selection of font (from Google fonts etc.) for translated text would be nice too..

Very nice extension! Congrats and thanks!

kappaflow commented 11 months ago

Actually, you can use translation to multiple languages even now. To do so, please follow “Multiple-instances setup for interviews” instruction. But I will consider to add this feature in a simpler way for Streaming Mode usage. Just keep in mind, if you are going to translate double amount of text you might need to use Google Translate (Script) for the second language to avoid reaching the free limits from Google. Also, you should consider to use Papago translation service for Korean language.

The typography and layout design in general was one of my main priorities while developing the extension. It uses Noto font, it’s the only font that is available for every language. It is designed for use cases where you need to display text in multiple languages together while preserving the same spacing and style. You can learn more here: https://fonts.google.com/noto. Therefore, the extension is bundled with Noto Sans font for all languages, but even when using the most size efficient format it takes 50Mb. Also, allowing users to choose fonts would break the text layout and might cause readability issues if they pick fonts that are not designed for body text. Basically, it is made for the same reason why you can’t change font in most applications where it's affects the UX.

Missingdev commented 3 months ago

Writing to support this idea. It would be great if the single translation window could support multiple output languages. Having to use a multi stream setup and running two translation is taking up a lot of screen space. It would be great if it were possible to consolidate down to one window, with both output languages tightly shown together.

kappaflow commented 3 months ago

@Missingdev thank you for the suggestion, but could you give a bit more details on how it could take less space, comparing to 2 windows where it is set to display only the translated text? Also, do I understand correctly that you mean the dual translation functionality for the Streaming Mode?

Missingdev commented 3 months ago

@Missingdev thank you for the suggestion, but could you give a bit more details on how it could take less space, comparing to 2 windows where it is set to display only the translated text? Also, do I understand correctly that you mean the dual translation functionality for the Streaming Mode?

Hello, yes. In this scenario, I am a streamer who is broadcasting in English, but also have Chinese and Korean viewers. In the below example, I have multi mode configured with OBS browser sources, one for English -> Chinese, and a second for English -> Korean. If I try to move them closer, they will overlap and no longer be readable. multibroadcast1

My suggestion was to allow the option with just one browser source for OBS, to include a translation from English -> Chinese & Korean. Like this example I manually edited: multibroadcast2

Perhaps I have missed an simpler solution for the same outcome I am suggesting.

Say one sentence, the translation shows both Chinese and Korean in the same window, more compact.

kappaflow commented 3 months ago

@Missingdev thank you for providing the mockups. Yes, I think its possible to add a such functionality specifically for the streaming mode, so you could add the second language in the same way as transcribed and the first translated text are displayed. Although it could be a bit difficult to read when you get multiple lines in a single chunk.

Missingdev commented 3 months ago

@Missingdev thank you for providing the mockups. Yes, I think its possible to add a such functionality specifically for the streaming mode, so you could add the second language in the same way as transcribed and the first translated text are displayed. Although it could be a bit difficult to read when you get multiple lines in a single chunk.

Great. If there are configuration options for how many history to keep, and the spacing between. It would allow some freedom to work with long sentences. Maybe only showing 3 messages total. With spacing between to allow for multi line responses.

Appreciate the fast responses!

kappaflow commented 3 months ago

Currently you can set the number of preserved translation in history by changing Configuration -> Expert Options -> Number of preserved final translations

By "multiline" I mean when the line of text (doesn't matter transcribed or translated) get split to the second line, its not a big deal when you have a single language, but when you have two it could be worse to read.

So ideally in case of using 2 languages, its better to use a setup like this one: https://lh3.googleusercontent.com/YQDV1RBZs2gd3QZkv4QfMqwHxT0Eg2mnunDd62-JOCLa7Lrz0g2cGE8nMwDg5qN8ONPl0H5VIl-SfpjK_SrZFBe8SMQ=s800-w800-h500 Additionally, maybe adjust Max number of letters per translation when Streaming Mode is turned on in Expert Options

kappaflow commented 3 months ago

@Missingdev when it comes to the terms, I am very busy with other projects at the moment, and the donations to this one barely cover the Streaming Mode functionality. However, I will try to add this functionality in a couple of months, because I know at least couple streamers who use my extension with 2 languages and the text gets overlapped as you've mentioned, and I don't like that.