search

fixie-ai / ai-jsx

The AI Application Framework for Javascript
https://docs.ai-jsx.com
MIT License
1.04k stars 79 forks source link

[Voice-QuickFix] Start speaking filler immediately #496

Closed farzadab closed 9 months ago

farzadab commented 9 months ago

I noticed that the filler sentences (or interjections as Peter calls them, which I'm not a total fan of), don't start until after a long pause and seemed to only start after the corpus response was ready, which defeats the purpose. The issue seemed to have been due to the filler not being recognized as a separate sentence which meant TTS would not be called.

This PR adds a newline between separate AssistantMessage responses from Fixie to solve this issue for Fixie calls.

Note that I did not investigate what happens in the other cases (LLM not from Fixie).

Logs before: notice no chunk is being requested from TTS for a long time. Screenshot 2023-11-15 at 9 47 08 AM

Logs after: TTS being called before actual content is ready, though it's still not easy to understand what's happening underneath when simply listening to it (there's no gap between filler TTS end and start of actual content). Screenshot 2023-11-15 at 9 43 31 AM

vercel[bot] commented 9 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
ai-jsx-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 15, 2023 7:19pm
ai-jsx-nextjs-demo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 15, 2023 7:19pm
ai-jsx-tutorial-nextjs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 15, 2023 7:19pm
ai-jsx-voice ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 15, 2023 7:19pm
farzadab commented 9 months ago

Sure, either works. I kinda see different assistant messages as being more separate (i.e. totally separate paragraphs even, otherwise why separate them) but I don't have a strong opinion here. I'll change it to a space.