When the chat room adds a text-to-speech model, the question and answer output is interrupted

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

Other

53.07k stars 7.74k forks source link

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I am using the normal chat room function. After I configured the text-to-speech and speech recognition functions for the chat room, an error message appears when I conduct normal Q&A, indicating that the status parameter does not exist. The picture below shows the error location. In the object audio_msg, status does not exist.This will then cause the entire output flow to be interrupted. QQ20241104-150121 The following picture is the error message: QQ20241104-150145

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

To resolve the issue where the status parameter does not exist in the audio_msg object, you should ensure that the audio_msg object is correctly instantiated and populated with the necessary attributes, including status. The audio_msg is expected to be an instance of AudioTrunk, which should have a status attribute.

Here's a relevant snippet from the code:

def _listen_audio_msg(self, publisher, task_id: str):
    if not publisher:
        return None
    audio_msg: AudioTrunk = publisher.check_and_get_audio()
    if audio_msg and audio_msg.status != "finish":
        return MessageAudioStreamResponse(audio=audio_msg.audio, task_id=task_id)
    return None

To fix the issue, you should:

Verify that the check_and_get_audio() method of the publisher object returns an AudioTrunk object with a status attribute.
Ensure that the AudioTrunk class is defined with a status attribute.

The AudioTrunk class is defined as follows:

class AudioTrunk:
    def __init__(self, status: str, audio):
        self.audio = audio
        self.status = status

If the AudioTrunk class is missing the status attribute, you need to add it to the class definition. If the check_and_get_audio() method is not returning an AudioTrunk object, you need to adjust the method to ensure it returns the correct object type ^[1]^[2].

To continue talking to Dosu, mention @dosu.

langgenius / dify