search

rhasspy / wyoming-satellite

Remote voice satellite using Wyoming protocol
MIT License
629 stars 92 forks source link

Implement push-to-talk functionality #5

Open chatziko opened 10 months ago

chatziko commented 10 months ago

Now that wyoming-satellite gives you more control over the satellite (similarly to esphome satellites), it would be very useful (and relatively easy to implement) to add "push-to-talk" functionality to it. I can imagine countless applications of starting a pipeline either by a physical button, or by various automations.

We could have a button or service on the HA that starts a pipeline directly at the STT stage, skiping wake word completely.

  1. When the satellite is disabled (and ideally idle, see also #4), the button/service could be used to manually run a single pipeline. The server could send RunSatellite with an argument to instruct the satellite to go directly to STT (a send back a corresponding RunPipeline(start_stage=STT) to the server).

  2. When the satellite is enabled, it still makes sense to use the button/service, to skip wake word and go straight to STT. Since there might be an active pipeline at this point, we should stop it and start a new one (or maybe completely stop the satellite and restart it the same way as (1) above).

I might give it a try at some point, but I'd like to hear @synesthesiam 's thoughts on this.

synesthesiam commented 10 months ago

I hadn't considered (1), though I wouldn't do anything when the satellite is disabled since the connection is broken then on purpose.

In fact, I don't think the satellite would even need to be aware if it's streaming. HA could simply kick the running pipeline from wake to STT. If the satellite is using VAD or local wake word detection, HA could just forward a Detection event and the satellites could move to the correct state.

(2) should pretty easy to do. My plan was to use the event service (--event-uri) in both directions for this. Right now, events always flow from the satellite to the event service. But a Detection event could go the other direction to inform the satellite that it should skip wake word and proceed to STT (via RunPipeline).

erikLundstedt commented 10 months ago

I would like to use a button connected to the GPIO-pins of the raspberry pi to "wake" it, instead of, or as an alternative to A wake-word, ive had some issues with stt hearing its own tts-"sorry I couldnt understand that"-response, leading to a long (I usually kill the script if it goes on for too long)loop

PsY4 commented 8 months ago

This combined with conversation_id (https://github.com/rhasspy/wyoming-satellite/issues/95) could lead to satellite asking questions/confirmations and directly proceed to STT for an answer...

chatziko commented 6 months ago

I gave it a try, see #144 and home-assistant/core#114291.

Clarification: my goal was to create a server-side "push-to-talk" button, which can be used to activate the satellite from the frontend, or automations, etc. Not a client-side button on the satellite side.

See home-assistant/core#114291 for design considerations.