Closed 11mail22 closed 7 years ago
shivasiddharth
commented
7 years ago What are the modifications that you did ? Also try commenting out the DETECT_DING line. You will anyway get a audio tone that i have created in assistant.py. DING is not required.
11mail22
commented
7 years ago I just reloaded the new files with actions.py and let it run totally unchanged, with same behavior! Maybe I gave to fast the commands in sequence and also mixed with google questions, and failed questions but it is not stable in comparison with google sdk and same setup.
Here the output, from the first trail fail of the latest snowboy src: Does this write underflow also occure on your side?
(env) pi@raspberryp GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:17
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:40
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:50
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:04
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:38
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:52
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:08
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:20
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:33
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:42
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:51
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:36:04
Traceback (most recent call last):
File "snowboy.py", line 58, in
shivasiddharth
commented
7 years ago it looks like a mic issue. try changing the mic.
11mail22
commented
7 years ago Just changed mic, with same behavior.
btw. I changed also the bing playing, as you proposed, without improvement.
It seems to be a protocol issue. After some working commands, it stucked in google assistant, meaning when saying "Alexa" google answered with "sorry I am Google Assistant" and it stays there until I say "stop". After this, alexa hotword detection is working again but stucks immediately in Google Assitant after the next command - each time. In this mode the crash comes very soon after some less commands.
This behavior I observed multible time.
With command I mean mix commands like "What time is it" and "trigger kitchen lights on".
How it can be ensured that Google listening is closed before entering snowboy, again?
11mail22
commented
7 years ago Now I can easly reproduce the crash: Simple and boring, hope you have an idea...
Saying "Alexa", Bing/Bing, "Stop" repeat this 9 times and it crashes, also with 4sec Pause after each word.
The "Stop" leads to END_OF_UTTERANCE and not to your "stop" implementation. Tried to add "return continue_conversation" in " if resp.event_type == END_OF_UTTERANCE:" without improvement.
Could there be any multiple instantiation somehow?
Sorry for piecemeal information.
(env) pi@raspberrypi GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:55:52
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:55:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:03
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:10
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:15
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:20
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:25
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:31
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:38
Traceback (most recent call last):
File "snowboy.py", line 58, in
(env) pi@raspberrypi GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:15:49
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:15:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:09
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:24
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:33
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:46
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:17:09
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:17:20
Traceback (most recent call last):
File "snowboy.py", line 58, in
shivasiddharth
commented
7 years ago Any piece of code can crash. There are so many factors which will cause a crash. That is why I created a service that would restart the snowboy if it crashed. if you run the snowboy as a service, even if it crashes it will restart. If you comment out the snowboy startup audio and run it as a service, the crashes will not be noticeable. And in the next update, i will try to deactivate snowboy when google assistant starts and activate it back when google assistant stops.
11mail22
commented
7 years ago I analyzed more in detail. The root cause of the crash is in the assistant.py after calling the Actions (and following things) which lead finally to return the function, but google sending in meanwhile the "not understand" answer, which will not be processed. I could imagine that a pipe got an overflow, after a while, even audio device is restarted by next request. I avoided this by clearing the audio_out buffer in case of actions and let process everything the same as with each google assist conversation (without pre-returning).
Now it runs stable so far:
My solution look like this (Action returns True if a valid command was found): if resp.result.spoken_request_text: usrcmd=resp.result.spoken_request_text if Action(str(usrcmd).lower()): self.logger.info('ACTION done') actionflag = True self.logger.info('Transcript of user request: "%s".',resp.result.spoken_request_text) self.logger.info('Playing assistant response.') if len(resp.audio_out.audio_data) > 0:
**if actionflag is False:
self.conversation_stream.write(resp.audio_out.audio_data)
else:
resp.ClearField('audio_out')** Another memory leak still exists. After each command the reserved memory grows about 64-128bytes. When running as python (not service) and stop if by CTRL+C, you will see the number of used commands by voice break and snowboy finished informations, so i seems a stack will be pushed and not fully released. Any idea? New issue? DEBUG:snowboy:finished. DEBUG:snowboy:detect voice break DEBUG:snowboy:finished. DEBUG:snowboy:detect voice break DEBUG:snowboy:finished. DEBUG:snowboy:detect voice break DEBUG:snowboy:finished. ....
shivasiddharth
commented
7 years ago I have an idea to modify the Ok-google main.py script to be triggered using snowboy. But i am too busy to work on it and moreover i just updated the project. So when i find time, i will update that.. Thanks for your suggestion regarding changes to assistant.py.. Regarding the memory usage i think it is due to the logs.
gassistpi-ok-google.service (main.py) running fully stable, but snowboy.service (GassistPi/snowboy/snowboy.py) crashes after a while. Traceback pointet always to missing device for playing the DETECT_DING when detecting keywords.
Using audio Jack and USB Mic od RasPi 3 with clean Stretch plus lirc and Util433 installed. Again, no issues with gassistpi-ok-google.service!
INFO:snowboy:Keyword 1 detected at time: 2017-10-08 20:30:59 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control TV Guide INFO:snowboy:Keyword 1 detected at time: 2017-10-08 20:31:12 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control TV channel 1 INFO:snowboy:Keyword 1 detected at time: 2017-10-08 20:31:25 Traceback (most recent call last): File "snowboy.py", line 58, in
sleep_time=0.03)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 172, in start
callback()
File "snowboy.py", line 41, in detected
snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 50, in play_audio_file
rate=ding_wav.getframerate(), input=False, output=True)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, kwargs)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 441, in init
self._stream = pa.open(arguments)
IOError: [Errno -9985] Device unavailable
Traceback (most recent call last):
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 80, in audio_callback
return play_data, pyaudio.paContinue
AttributeError: 'NoneType' object has no attribute 'paContinue'
Exception AttributeError in <function _remove at 0x76bf78b0> ignored
little bit different, but same result: I stucked in Google Assist instead of listening to hotword, this is why Alexa is recognized. No sound was played but MIC seems to work. After two tries and wait time, due to audio device the assistant.py crashed. System is just loaded <5% during waiting and max. 25% while processing
INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:03:39 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control light off INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:03:48 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control light on INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:03:56 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control light off INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:04:07 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: Alexa INFO:assistant:Expecting follow-on query from user. WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1396) INFO:assistant:Finished playing assistant response. INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:Finished playing assistant response. INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:04:34 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: Alexa INFO:assistant:Expecting follow-on query from user. WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) WARNING:root:SoundDeviceStream write underflow (size: 1600) INFO:assistant:Finished playing assistant response. INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:Finished playing assistant response. INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:06:37 INFO:assistant:Recording audio request. INFO:assistant:End of audio request detected INFO:assistant:TEXT: control lights on INFO:snowboy:Keyword 1 detected at time: 2017-10-08 21:06:53 Traceback (most recent call last): File "snowboy.py", line 59, in
sleep_time=0.03)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 172, in start
callback()
File "snowboy.py", line 43, in detected
gassist.assist()
File "/home/pi/GassistPi/snowboy/assistant.py", line 125, in assist
flush_size=self.audio_flush_size
File "/home/pi/env/local/lib/python2.7/site-packages/googlesamples/assistant/grpc/audio_helpers.py", line 187, in init
blocksize=int(block_size/2), # blocksize is in number of frames.
File "/home/pi/env/local/lib/python2.7/site-packages/sounddevice.py", line 1491, in init
**_remove_self(locals()))
File "/home/pi/env/local/lib/python2.7/site-packages/sounddevice.py", line 1017, in init
'Error opening {0}'.format(self.class.name))
File "/home/pi/env/local/lib/python2.7/site-packages/sounddevice.py", line 2671, in _check
raise PortAudioError(msg)
sounddevice.PortAudioError: Error opening RawStream: Device unavailable