Closed yukdumboobumm closed 4 years ago
Can it be related to disabled command
?
Dummy Disables voice command listening. https://rhasspy.readthedocs.io/en/latest/command-listener/#dummy
I have HA as well and it works great in both ways
@frkos are you using the latest (12/23/2019) docker image?
The config I listed is for my server instance...the client machine does the voice detection and wake-word.
both included below for reference: Server:
{
"command": {
"system": "dummy"
},
"handle": {
"system": "hass"
},
"home_assistant": {
"access_token": "REMOVED",
"url": "http://192.168.50.2:8123"
},
"microphone": {
"system": "dummy"
},
"rhasspy": {
"listen_on_start": false
},
"sounds": {
"system": "dummy"
},
"text_to_speech": {
"system": "dummy"
}
}
Client:
{
"intent": {
"remote": {
"url": "http://192.168.50.2:12101/api/text-to-intent"
},
"system": "remote"
},
"microphone": {
"pyaudio": {
"device": "2"
}
},
"sounds": {
"aplay": {
"device": "default:CARD=seeed2micvoicec"
}
},
"speech_to_text": {
"remote": {
"url": "http://192.168.50.2:12101/api/speech-to-text"
},
"system": "remote"
},
"text_to_speech": {
"system": "picotts"
},
"wake": {
"snowboy": {
"model": "snowboy/computer.pmdl",
"sensitivity": "0.5"
},
"system": "snowboy"
}
}
The client is successfully sending the WAV file to the server, but the server is not POSTing the intent to the HA api.
I'm using the latest 2.4.14.5 But I have all-in-one installation (hassio addon)...
According to the document:
(A text/plain response with the transcription is expected back. An additional profile query argument is sent with the current profile name, so the POST URL is effectively something like http://remote-server:12101/api/speech-to-text?profile=en)
So if I'm right, we have this logic: 1) client sends wav to server 2) server converts it to text and sends back to client 3) client sends received text back to server 4) server converts text to intent and process
But when you send text manually, it works... so 3) and 4) is ok Could you check your clients logs? in point 2) it should receive text from the server
As for me, rpi 3B+ is powerful enough to be a server, not just a clinet =)
Client-side:
[DEBUG:139461741] SnowboyWakeListener: loaded -> listening
[DEBUG:139461739] DialogueManager: ready -> asleep
[INFO:139461738] DialogueManager: Automatically listening for wake word
[DEBUG:139461735] DialogueManager: handling -> ready
[DEBUG:139461734] WebSocketObserver: {"entities": [{"end": 11, "entity": "name", "raw_end": 21, "raw_start": 5, "raw_tokens": ["living", "room", "wall"], "raw_value": "living room wall", "start": 5, "tokens": ["bulb_3"], "value": "bulb_3"}, {"end": 14, "entity": "state", "raw_end": 24, "raw_start": 22, "raw_tokens": ["on"], "raw_value": "on", "start": 12, "tokens": ["on"], "value": "on"}], "intent": {"confidence": 1.0, "name": "ChangeLightState"}, "raw_text": "turn living room wall on", "raw_tokens": ["turn", "living", "room", "wall", "on"], "recognize_seconds": 0.0007333040121011436, "siteId": "default", "slots": {"name": "bulb_3", "state": "on"}, "speech_confidence": 1, "text": "turn bulb_3 on", "time_sec": 0.0028085708618164062, "tokens": ["turn", "bulb_3", "on"], "wakeId": "snowboy/computer.pmdl"}
[DEBUG:139461732] DialogueManager: recognizing -> handling
[DEBUG:139461730] DialogueManager: {'entities': [{'end': 11, 'entity': 'name', 'raw_end': 21, 'raw_start': 5, 'raw_tokens': ['living', 'room', 'wall'], 'raw_value': 'living room wall', 'start': 5, 'tokens': ['bulb_3'], 'value': 'bulb_3'}, {'end': 14, 'entity': 'state', 'raw_end': 24, 'raw_start': 22, 'raw_tokens': ['on'], 'raw_value': 'on', 'start': 12, 'tokens': ['on'], 'value': 'on'}], 'intent': {'confidence': 1.0, 'name': 'ChangeLightState'}, 'raw_text': 'turn living room wall on', 'raw_tokens': ['turn', 'living', 'room', 'wall', 'on'], 'recognize_seconds': 0.0007333040121011436, 'siteId': 'default', 'slots': {'name': 'bulb_3', 'state': 'on'}, 'speech_confidence': 1, 'text': 'turn bulb_3 on', 'time_sec': 0.0028085708618164062, 'tokens': ['turn', 'bulb_3', 'on'], 'wakeId': 'snowboy/computer.pmdl'}
[DEBUG:139461721] urllib3.connectionpool: http://192.168.50.2:12101 "POST /api/text-to-intent?profile=en&nohass=True HTTP/1.1" 200 682
[DEBUG:139461705] urllib3.connectionpool: Starting new HTTP connection (1): 192.168.50.2:12101
[DEBUG:139461691] DialogueManager: decoding -> recognizing
[DEBUG:139461690] DialogueManager: turn living room wall on (confidence=1)
[DEBUG:139461681] urllib3.connectionpool: http://192.168.50.2:12101 "POST /api/speech-to-text?profile=en HTTP/1.1" 200 24
[DEBUG:139460831] urllib3.connectionpool: Starting new HTTP connection (1): 192.168.50.2:12101
[DEBUG:139460796] RemoteDecoder: POSTing 221804 byte(s) of WAV data to http://192.168.50.2:12101/api/speech-to-text
[DEBUG:139460795] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=seeed2micvoicec', '/usr/share/rhasspy/etc/wav/beep_lo.wav']
[DEBUG:139460793] DialogueManager: awake -> decoding
[DEBUG:139460789] WebrtcvadCommandListener: listening -> loaded
[DEBUG:139460786] WebrtcvadCommandListener: Voice command finished
[DEBUG:139453827] WebrtcvadCommandListener: Voice command started
[DEBUG:139453392] WebrtcvadCommandListener: loaded -> listening
[DEBUG:139453372] WebrtcvadCommandListener: Will timeout in 30 second(s)
[DEBUG:139453371] APlayAudioPlayer: ['aplay', '-q', '-D', 'default:CARD=seeed2micvoicec', '/usr/share/rhasspy/etc/wav/beep_hi.wav']
[DEBUG:139453367] SnowboyWakeListener: listening -> loaded
[DEBUG:139453364] DialogueManager: asleep -> awake
[DEBUG:139453363] DialogueManager: Awake!
[DEBUG:139453358] SnowboyWakeListener: Hotword(s) detected: ['snowboy/computer.pmdl']
and Server Side:
[INFO:93975] quart.serving: 192.168.50.30:50756 POST /api/text-to-intent 1.1 200 682 6972
[DEBUG:93972] __main__: {"intent": {"name": "ChangeLightState", "confidence": 1.0}, "entities": [{"entity": "name", "value": "bulb_3", "raw_value": "living room wall", "start": 5, "raw_start": 5, "end": 11, "raw_end": 21, "tokens": ["bulb_3"], "raw_tokens": ["living", "room", "wall"]}, {"entity": "state", "value": "on", "raw_value": "on", "start": 12, "raw_start": 22, "end": 14, "raw_end": 24, "tokens": ["on"], "raw_tokens": ["on"]}], "text": "turn bulb_3 on", "raw_text": "turn living room wall on", "recognize_seconds": 0.0007333040121011436, "tokens": ["turn", "bulb_3", "on"], "raw_tokens": ["turn", "living", "room", "wall", "on"], "speech_confidence": 1, "slots": {"name": "bulb_3", "state": "on"}, "wakeId": "", "siteId": "default", "time_sec": 0.0028085708618164062}
[INFO:93932] quart.serving: 192.168.50.30:50754 POST /api/speech-to-text 1.1 200 24 829149
[DEBUG:93928] PocketsphinxDecoder: turn living room wall on
[DEBUG:93928] PocketsphinxDecoder: Transcription confidence: 0.0012184474722329018
[DEBUG:93924] PocketsphinxDecoder: Decoded WAV in 0.7617373466491699 second(s)
[DEBUG:93160] PocketsphinxDecoder: rate=16000, width=2, channels=1.
[INFO:79785] quart.serving: 192.168.50.30:50752 POST /api/text-to-intent 1.1 200 158 7081
I'm having a hard time piecing together the log, but it seems like the Client is POSTing to both speech-to-text and text-to-intent, and receiving responses to both. Although the text-to-intent response appears twice, once in DialogueManager and once in WebSocketObserver.
Agreed about the 3b+ being able to do the work,. I'm using a Server / client setup because my server is located in a cabinet on the other side of the house. If I can't get this to work, I'll just run everything on the client and then send the intent from that machine. Still, it'd be neat to get this setup working.
Finally understood what you were trying to say.
I added Intent Handling to the Client (in addition to the server), so now the intent is still transcibed by the Server, but the handling to HA is done with the Client. Everything works fine. I think there is some ambiguity in the docs that could be cleared up to make this more obvious. Will submit a request there. Thanks for the help and Merry Xmas!
-JD
HomeAssistantIntentHandler POSTs correctly when using the "Get Intent" + "Send to Home Assistant" functionality on "Speech" tab:
But not when speaking:
I'm also listening on my HA server for events. I see the event when using text-to-intent, but nothing from speech-to-intent.
Using a server / client setup as outlined in your docs
Server (also where my HA instance is):
Client:
Is there a more verbose log setting that I can enable to help debug?