WatchOS text-to-speech-failed

[miumer]

iOS device model, version and app version

Model Name: Apple watch Ultra Software Version: watchOS 10.6.1; iOS 17.6.1 App version: 2024.8.1

Home Assistant Core Version 2024.6.2

When opening Assist in my apple watch it is very unreliable with speech-to-text and answers. Sometimes it thinks for a very long time, then gives a red message: "speech-to-text failed", but then still prints underneath the text i spoke, so it seems it actually didn't fail. Sometimes nothing happens after that--no answer, but then sometimes when exactly the same thing is displayed: "speech-to-text failed" it does answer to the question so for some reason it only works sometimes but bugs out with "speech-to-text failed" most of the time. Everything works on iphone. All widgets and going to assist through the app etc. In the screenshots below i actually got an answer once after the "speech-to-text failed" and then my question registered.

To Reproduce Im using "extended OpenAI Conversation" addon for Openai integration as the conversation agent to Home Assistant. To reproduce add this conversation agent to a new Assist Pipeline and run the Assist app on your watch watch asking something about a sensor that has been exposed to the voice assistants.

Expected behavior Assist asnwers the same way my iphone does with the "Assist in App" widget for example

Screenshots

[bgoncal]

Assist on Apple Watch is still in beta, we just released a new version of the App (2024.9), give it a try. Heads up that this version also rebuilds the way Watch is configured, after updating it, go to companion app settings and under Apple Watch, look for configuration, create yours and save it.

[miumer]

Seems to work very well and good responsiveness. Thank you a lot. Streamlines the entire Home Assistant experience for me.

Another feeback in terms of the watch and Assist that maybe you can concider in the next iteration is that i have some sensors set up that can fetch quite a long response. Like news headlines, or electricity prices at different hours. What happens with the watch is that when i ask for the asnwer and then lower my hand the voice fades out. Its probably because my watch thinks it needs to go the idle mode when i lower my hand.

Sidenote: Text color for actions doesn't work in the new version i think. Defaults to white.

[miumer]

After using it for some days. Im still experiencing some slowness and lag in Assist registering what i say. Especially when i start the app for the first time after closing it from the background.

Its weird because there seems to be two opposite problems on different times. Sometimes when i open the app and speak to Assist it doesn't stop listening when i stop speaking. I suspect thats because it doesnt even register that im speaking. But other times, most of the time when it works well, when im asking something long it stops listening mid sentence so i cant finish what i want to ask or say unless i do it really fast in a very short sentence.

But still, most of the time it is working much better than before

EDIT: Found that it works better if i manually tap the watchface when i finish my command/sentence...