search

Finity-Alpha / OpenVoiceChat

Have a natural voice conversation with an LLM
http://www.finityalpha.com/OpenVoiceChat/
Apache License 2.0
224 stars 34 forks source link

Refine Interruption Handling #43

Closed thy-hoang-nn closed 1 month ago

thy-hoang-nn commented 3 months ago

Hi, Thank you for your contribution to the voice-based conversational engine. I'm encountering an issue while reimplementing your code. Specifically, the bot is designed to operate in a streaming mode, but the problem arises when it starts speaking the first processed sentence, the recording then gets interrupted and it doesn’t continue with the rest of the response.. How can fix the issue ?

fakhirali commented 3 months ago

Hmm, interruptions happen when the bot is speaking and the user says something in between. This stops the bot speaking (audio being played) and lets the user speak. If you're running main.py try adding the following and see if the error persists. The following line will disable interruptions.

    api_key = os.getenv('DEEPGRAM_API_KEY')
    # ear = Ear_deepgram(silence_seconds=2, api_key=api_key)
    ear = Ear_hf(silence_seconds=2, device=device)

    # ----The following line will disable interruption handling----
    ear.interrupt_listen = lambda x: ""

    load_dotenv()

    chatbot = Chatbot_gpt(sys_prompt=llama_sales)

p.s Also since the lib is in alpha rn, would recommend installing using pip install git+https://github.com/Finity-Alpha/OpenVoiceChat.git

thy-hoang-nn commented 1 month ago

Thanks for your help