joshuar / go-hass-agent

A Home Assistant, native app for desktop/laptop devices.
MIT License
135 stars 11 forks source link

[FEATURE REQUEST] Media player capabilities and voice controls using mic #234

Open andzejsp opened 1 month ago

andzejsp commented 1 month ago

Is your feature request related to a problem? Please describe. Im looking for a way to use the pc to control my home assistant using voice controls STT and receive back responses TTS and jingle on the pc (media player)

Describe the solution you'd like Similar to what ESPhome but with pc power. ESP devices are weak, unreliable. PC is much stable and in my case always running. I would like for it to record mic while its on to check for openwakeword, if wake word is triggered then it sends a jingle to my speakers that are connected to pc and starts listening for my commands. After the command is done its sent to assist and then it returns approval or something to the speakers on the pc. I would use this to for example lower volume on the pc when something is playing or interact with home assistant when im there.

Describe alternatives you've considered No alternatives have been found so far.

Additional context This would use the existing devices and would give more quality of life to people with disabilities without buying more unstable devices like ESP, raspberry etc..

joshuar commented 1 month ago

Hey @andzejsp that is a super interesting idea.

So from what I am understanding:

From what is available, I think I can do the above, but I'd need to look into some kind of text to speech engine (which shouldn't be a big hurdle).

The wake word functionality might be trickier. I'll have to look into that.

I think this all is possibly doable. It's a very cool idea!

andzejsp commented 1 month ago

Hey, dont go spending your time inventing new stuff. There are already things figured out for this.

As for wake word - Wyoming satellite Home assistant has Wyoming protocol integration meaning STT AND TTS capabilities.

As for media player i found that you can use vlc with telnet to send TTS responses.

Im still Jerry rigging these together with my limited coding skills.

https://github.com/rhasspy/wyoming-satellite

https://www.home-assistant.io/integrations/wyoming/

https://alanedwardes.com/blog/posts/vlc-remote-control-home-assistant/

Im struggling with Wyoming satellite because of python intricacies..

andzejsp commented 3 weeks ago

Actually vlc is not reliable media player. MPD (media player demon) server is much more reliabe (on arch) but i havent had luck setting it up on popos. It set up alright but no sound is playing when i send audio - tts - from HA to MPD on the popOS device.

And i have set up that wyoming satellite on arc too - works great in testing but again.. have issues on popOS :D...