enricoros / big-AGI

Generative AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. It features AI personas, AGI functions, multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
https://big-agi.com
MIT License
5.59k stars 1.29k forks source link

[BUG] Incorrect accent in Elevenlabs voice output when non-English language is selected #649

Open rwb-truelime opened 1 month ago

rwb-truelime commented 1 month ago

Description

When selecting a non-English language (specifically Dutch) for voice output using Elevenlabs, the generated speech still has a strong English accent instead of the expected Dutch accent. This issue persists even when a Dutch voice is explicitly selected.

Steps to reproduce:

  1. Open big_AGI
  2. Navigate to the language selection settings
  3. Choose Dutch as the language
  4. Select the Dutch voice (voice_id: YUdpWWny7k5yb4QCeweX) for Elevenlabs output
  5. Generate voice output for Dutch text

Expected behavior: The voice output should have a proper Dutch accent matching the selected Dutch voice.

Actual behavior: The voice output has a strong English accent, despite the Dutch language and voice selection.

Additional information:

Please investigate why the accent is not changing according to the selected language and voice, and implement a fix to ensure the correct accent is applied for non-English language selections.

Device and browser

Device: Dell XPS 15 9520 OS: Microsoft Windows 11 Enterprise, Version 10.0.22631 Build 22631 Browser: Microsoft Edge 129.0.2792.52 (Official build) (64-bit)

Additional details:

Processor: 12th Gen Intel(R) Core(TM) i7-12700H, 2300 MHz, 14 core(s), 20 logical processor(s) RAM: 32.0 GB System Type: x64-based PC BIOS Version: Dell Inc. 1.25.0, 11-7-2024 Language: Dutch (based on the system information being in Dutch)

Screenshots and more

No response

Willingness to Contribute

darthalex2014 commented 1 month ago

I'm having the same issue. I even switched to the eleven_turbo_v2_5 model (it supports Russian, unlike eleven_multilingual_v1), but it still speaks with an English accent.

enricoros commented 1 month ago

Thanks @rwb-truelime -- does it do the same thing if the language is changed inside Preferences? image

It's possible we are forcing this language in the app. @darthalex2014 does eleven_turbo_v2_5 detect the language automatically? If so then we don't need the UI selector for the language anymore, and we'd rather just call the latest model.

rwb-truelime commented 1 month ago

Thanks @rwb-truelime -- does it do the same thing if the language is changed inside Preferences? image

It's possible we are forcing this language in the app. @darthalex2014 does eleven_turbo_v2_5 detect the language automatically? If so then we don't need the UI selector for the language anymore, and we'd rather just call the latest model.

Yes, that is the place I've set the language to Dutch. Thanks for looking into this!

zoollcar commented 1 month ago

I test in Chinese, Use the following code to switch to eleven_turbo_v2_5, I think it can detect the language automatically

src\modules\elevenlabs\elevenlabs.server.ts

    const body: ElevenlabsWire.TTSRequest = {
      text: text,
      ...({ model_id: 'eleven_turbo_v2_5' }),
    };

It sounds much better than eleven_multilingual_v1 or eleven_multilingual_v2, It still with an English accent but acceptable.