Sonus.stop not stopping Google Speech

[jaumard]

From what I try here Sonus.stop just stop to record but doesn't cancel any Google Speech request if there some pending https://github.com/evancohen/sonus/blob/master/index.js#L153

I try to put a basic timeout because sometime it stay in listen state and never end... Basically I do like this :

Sonus.stop(this.sonus)
Sonus.start(this.sonus)

But Google Speech still streaming :(

[evancohen]

You can't run Sonus.stop(sonus) and Sonus.start(sonus) on the same sonus object. See the API Docs on Stopping recognition.

I think what you're actually looking for here is .pause/.resume.

That said, Sonus.stop(sonus) should kill a recognition process because audio recording stops at that line of code you mentioned. Note sure why this wouldn't happen unless there's a GCS bug.

[jaumard]

How I miss that thanks ! What I'm trying to achieve is to put a timeout after some time because sonus I don't know why sometime doesn't trigger the final-result and stay in inconsistent state so I want to reset the recording, is it possible ?

[evancohen]

Similar to your other issue regarding the internal state. cloudSpeechRecognizer.listening only lives in the scope of Sonus and can't be accessed through the API. I think this is the way that it should be, but by fixing #57 this should also be resolved.

If this is still another independent issue I could create a hacky API call to manually reset the internal state, but I'd really rather not - it makes me feel dirty even suggesting it ;)

[jaumard]

I don't think it's relate to #57 as sometime it detect the hot work and stay forever in that state (no error or anything)

Instead of a hacky solution to reset the internal state maybe it will be better to add a nice API to allow a timeout for the listening part. I think it's a good idea. If you point me to the right way I can make a PR for this. But I agree let's wait for #57 first and see if it's better after.

[evancohen]

As I mentioned in the other issue, I'm going to dig into the API again and see if there's a good way to examine the state of the streaming recognizer (instead of having our own internal state).

[jaumard]

I still think that a timeout function will be helpful because I think if grpc doesn't close the connexion because of a low bandwidth (but not low enough to have an error lol) then it stay in weird state

[jaumard]

Any progress on those 2 issues ? Anything I can do to help @evancohen ?

[evancohen]

I've got a new job so I haven't had the time to really get cracking on this. I should have some time on Wednesday. I'm blocking some time out on my calendar to take a look :)

[evancohen]

Ok, so I did get the chance to peak this. It looks like there's a way to modify the parameters that are passed to streamingRecognize and on to gax.CallOptions to modify timeout, number of retries, and even Backoff - so it's totally possible.

Right now we use createRecognizeStream. The newer version of GCS uses streamingRecognize, so I'm going to have to update Sonus before this can be exposed. It should be a pretty trivial update so I should be able to knock it out this weekend :)

Thoughts?

[jaumard]

Thanks a great news @evancohen ! Just take a look at the doc and it's look like exactly what we need here in order to stabilize sonus ! :D The backoff capabilities can be really interesting for people that want to use the long recognition with partial results 👍

But still no other way than the internal state to know if it's in listening state ? Still need to be put to false on end event if not.

[evancohen]

This might be fixed by updating the library. I haven't looked at the current version that we're using, but the most recent Node GCS library destroys the stream which should cause Sonus to update its internal listening state correctly (not sure how this is different with what we're doing today, but let's try updating first and seeing if that fixes things)

[jaumard]

Agree let try the last version with the timeout changes and see if it's all good :) I'm available this weekend to test from a branch to not release it directly this weekend if you prefer

[evancohen]

Thanks! It'll be good to have another pair of eyes on it. I'll just spin out a pre-release for the package. I'm going to want to test it across devices and that'll be the easiest way.

[jaumard]

Fine for me ! :) Let me know when it's published and I'll test it

[evancohen]

Ok, so I updated Sonus to use the latest version of Google Cloud Speech - which actually took a bit of re-writing because a bunch of things have changed. You can test this new version with:

npm install sonus@next

I'm hoping that just by upgrading we can cure some of the strange behavior. I haven't exposed the CallOptions I mentioned earlier (at least not yet) because it looks like there's an internal timer built into the API now that times out a request automatically. Can you give this a shot?

[jaumard]

Cool let's try it :D I'll let you know how it goes, thanks @evancohen

[jaumard]

Just tested a bit and the problem is still here, once I have this error:

error:  
{ code: 11,
  message: 'Audio data is being streamed too slow. Please stream audio data approximately at real time.',
  details: [] }
error:  
{ streamingError: 
   { code: 11,
     metadata: { _internal_repr: { 'content-disposition': [ 'attachment' ] } } } }
error:  
{ code: 11,
  message: 'Audio data is being streamed too slow. Please stream audio data approximately at real time.',
  details: [] }
error:  
{ streamingError: 
   { code: 11,
     metadata: { _internal_repr: { 'content-disposition': [ 'attachment' ] } } } }

Then the next hotword put sonus into listening mode but nothing append, never :( I have to restart the project to make it work again. Did you reset isListening on end event on this version ?

[jaumard]

Also is it possible to force sonus to stop GSpeech recognition on error ? Because on my project I have made the assumption that the stream is closed when I receive an error event but it's not the case :( or having a new event that say the the online recognition stream is close...

It really visible in my case because I have light for each state, listening is green, error is red and idle is rainbow. I go (and want to) go back to idle/rainbow after an error and close the stream.

So currently I go back to idle/rainbow but potentially sonus is still listening :/

[jaumard]

I also think that now after a final-result event the stream is not closed, because I have put some TTS answer after a final-result and it trigger another final-result with what the TTS said :/ I didn't have this with the previous version

[evancohen]

I'm not sure how I can fix the audio streaming speed issue, but I can certainly take a stab at making the internal state reset after an error. My only concern here is that even if the audio is being streamed slowly we may still actually get some recognition (and therefore some weird behavior). I'll have time to rev this again over the weekend.

[mrdago]

Hi, I've a similar issue as @jaumard. Sometimes sonus is recording but doesn't trigger the final-result and stay in inconsistent state. Over the next weekend I'll monitor the behaviour using the the pre-release I've just installed. A first try looks very promising!!

[evancohen]

Had a few spare min and published a new prelease sonus@0.1.9-1. Your funky error state should be fixed now: https://github.com/evancohen/sonus/commit/bfafe3bd3a750275c09c4d85426add26a7c36fb9

Give that a shot and let me know if you experience anything weird :)

[evancohen]

This issue is also fixed with #67