Text To Speech Issue [auto play]

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

https://dify.ai

Other

50.8k stars 7.3k forks source link

Text To Speech Issue [auto play] #10076

Open asrguru20 opened 6 days ago

asrguru20 commented 6 days ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.2

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

enable the TTS with auto play enabled feature in a agent chat bot

✔️ Expected Behavior

When You ask the chat bot which has TTS feature enabled with auto play it should automatically start to play the audio smoothly

❌ Actual Behavior

while generating text in chat bot it starts to generate audio of that text simultaneously most of the case it has at least one delay or lag but mostly two lags at the beginning of the audio

dosubot[bot] commented 6 days ago

I found a similar issue related to audio lag when using the Text To Speech feature with auto play enabled in a self-hosted Dify version 0.10.2 chat bot. To resolve this, you can adjust the TTS_AUTO_PLAY_TIMEOUT and TTS_AUTO_PLAY_YIELD_CPU_TIME settings in the api/constants/tts_auto_play_timeout.py file.

The current settings are:

TTS_AUTO_PLAY_TIMEOUT = 5
TTS_AUTO_PLAY_YIELD_CPU_TIME = 0.02

Consider reducing the TTS_AUTO_PLAY_TIMEOUT value to decrease the delay before the audio starts playing. Additionally, you might want to experiment with the TTS_AUTO_PLAY_YIELD_CPU_TIME to optimize CPU yield time for smoother playback. Adjust these values incrementally and test the performance to find the optimal configuration for your setup ^[1].

To continue talking to Dosu, mention @dosu.

asrguru20 commented 6 days ago

what is suggested values for those two variables ? should I increase the TTS_AUTO_PLAY_TIMEOUT and TTS_AUTO_PLAY_YIELD_CPU_TIME ?

dosubot[bot] commented 6 days ago

The suggested values for the variables are:

TTS_AUTO_PLAY_TIMEOUT: 5
TTS_AUTO_PLAY_YIELD_CPU_TIME: 0.02

There is no specific information provided on whether you should increase them. It might be beneficial to experiment with these values to see how they affect performance in your specific environment. Adjust them incrementally and test the results to find the optimal configuration for your setup ^[1].

To continue talking to Dosu, mention @dosu.

asrguru20 commented 6 days ago

is the TTS_AUTO_PLAY_TIMEOUT for delay time before auto play TTS ?