[Bug] Whisper uses only english

[Martin-Hausleitner]

📦 Environment

Official Preview

📌 Version

1.16.2

💻 Operating System

Windows

🌐 Browser

Chrome

🐛 Bug Description

Whisper uses only english but it should also uses other langs.

📷 Recurrence Steps

add a setting to change that

🚦 Expected Behavior

No response

📝 Additional Information

No response

[lobehubbot]

👀 @Martin-Hausleitner

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible. Please make sure you have given us as much context as possible.\ 非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot[bot]]

@Martin-Hausleitner Hello there! I'm here to help you with any bugs, questions, or contributions you might have. Let's squash this bug together! Dosu

The Whisper language setting can be configured in the tts section of the JSON configuration file. Specifically, it is set under tts.openAI.sttModel with the value "whisper-1". Additionally, the sttLocale setting under defaultAgent.config.tts can influence the language settings for speech-to-text operations:

"tts": {
  "openAI": { "sttModel": "whisper-1", "ttsModel": "tts-1" },
  "sttAutoStop": true,
  "sttServer": "openai"
},
"defaultAgent": {
  "config": {
    "tts": {
      "showAllLocaleVoice": false,
      "sttLocale": "auto",
      "ttsService": "openai",
      "voice": { "openai": "alloy" }
    }
  }
}

To change the language settings, you can modify the sttLocale value to the desired language code instead of auto ^[1][2][3].

Additionally, there is documentation in the code that explains how to change the language settings for Whisper. Specifically, the code snippet includes a comment and logic for automatically switching the language:

// auto switch language
const language = userGeneralSettingsSelectors.config(get()).language;
if (language === 'auto') {
  switchLang('auto');
}

This indicates that if the language setting is set to 'auto', the switchLang function will be called with 'auto' as the parameter to handle the language switching ^[4].

The switchLang function is implemented in the file src/utils/client/switchLang.ts. It handles language switching for Whisper by determining the language to switch to based on the provided locale parameter. If the locale is set to 'auto', it uses the browser's default language (navigator.language). It then changes the language using the changeLanguage function from the i18next library and updates the lang attribute of the document.documentElement to reflect the new language ^[5].

To continue talking to Dosu, mention @dosu.