Google Speech error stop Sonus to work

[jaumard]

When I have this error return by sonus, then nothing work anymore (no keyword detection) I have to restart my project to have it working again :(

error:  
{ code: 11,
  message: 'Audio data is being streamed too slow. Please stream audio data approximately at real time.',
  details: [] }
error:  
{ streamingError: 
   { code: 11,
     metadata: { _internal_repr: { 'content-disposition': [ 'attachment' ] } } } }

[evancohen]

What's your internet connection speed like? Off the top of my head there could be two things happening here, maybe you can help me figure out which one it is:

• Google Cloud Speech/gRPC suffers some unrecoverable connection issue which causes audio streaming to stop working
• Sonus is unable to recover from this error gracefully

Ideally Sonus shouldn't have to re-create any of the connection objects to recover from an error like this, but it's possible that it's getting hung up on something when an error is triggered.

[jaumard]

Maximum should be 8Mo lol but when I'm downloading a film or tv show then there no bandwidth left for other stuff and everything became really slow :( It's the same error I had before (coming from gRPC) but now it doesn't kill the entire project, sonus retrieve the error from gRPC and give it to me correctly with error event. But once I receive it I'm not able to trigger the hot word anymore.

I'll say it can be the second option because it's Google Cloud Speech/gRPC who is failing but even the hot work it's not working anymore so maybe the the audio stream is closed with this error (and then do like the Sonus.stop and need a full creation of sonus object to make it work...

[evancohen]

Ok, that actually gives me a much better understanding of what's going on! My guess is that when the error event fires the 'END_OF_SINGLE_UTTERANCE' speechEventType is never returned by the data event (so we aren't updating our internal cloudSpeechRecognizer.listening state.

In index.js#L40 we should be doing something like

recognitionStream.on('error', err => {
    cloudSpeechRecognizer.emit('error', err)
    cloudSpeechRecognizer.listening = false
})

This of course assumes that an error event won't be proceeded by an end of utterance event, but that's probably an ok assumption. We could also add a special case to check the error code and only set state for codes that we know won't be followed by data events, but maybe that's just equally hacky...

Can you make this change locally, test it, and report back? If it works I'll create an update ASAP.

[jaumard]

Yes I'll try that asap and let you know if it fix the problem :)

I wanted to know, what append when you say the hot word when it's already listening ? Does it restart the listening or just ignore the hot word ? Because sometime when it freeze on listening state I try to say again the hot word but it doesn't work (but can see on logs that the hot word it detected)

[jaumard]

@evancohen I have a great news and a bad news !

The good news is I manage to reproduce my very low bandwidth and with the snippet I'm able to have the hot word (and speech detection) working again :)

The bad news is that during my tests the #56 still append :(

[evancohen]

That's ok! We can treat that as a separate issue. To answer your question, if you say the hotword while detection is runing, sonus will ignore it (and the hotword would be detected as a part of the utterance).

I'm going to dig into their documentation and make sure that we can safely make this change without changing the behavior of Sonus. I'll post back here what I find.

[evancohen]

Ok, something else for you to test if you don't mind. Revert that change from before and try adding a new event handler for the end of the stream, so it would look something like:

recognitionStream.on('error', err => cloudSpeechRecognizer.emit('error', err))
recognitionStream.on('end', () => {cloudSpeechRecognizer.listening = false})

[jaumard]

Cool thanks a lot ! That will make sonus much more stable on my side :) sonus will ignore it (and the hotword would be detected as a part of the utterance). from my test the hot word event is still fired (maybe ignored but in that case it should fire the event).

With this code when the error occurred the hot word is detected but then the #56 append (each time) :( maybe those two are related finally... Difficult to know. With the first code I'm able to have multiple time the error and then a correct answer (but sometime #56 append too) Sorry if I'm not helping a lot lol but anyway it's look like recognitionStream.on('end', () => {cloudSpeechRecognizer.listening = false}) should be part of the sonus code

[jaumard]

As it depend of the network is kind of hard to test and reproduce :/ maybe #56 and #57 are related to network because they append together, the other cause will be that one trigger the other but can't be sure...

[evancohen]

This should be fixed with #67