Error event and crash

[jaumard]

Hey,

I use the last version v0.1.8 and now once the hotword is detected I have an error (even if the sentences are correctly returned). This is the error:

error:  
{ code: 11,
  message: 'Audio data is being streamed too slow. Please stream audio data approximately at real time.',
  details: [] }
error:  
{ streamingError: 
   { code: 11,
     metadata: { _internal_repr: { 'content-disposition': [ 'attachment' ] } } } }

Then it crash with:

_stream_writable.js:383
  cb();
  ^

TypeError: cb is not a function
    at afterWrite (_stream_writable.js:383:3)
    at onwrite (_stream_writable.js:374:7)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

More info: OS: Mac OSX Node: v7.10.0

On version 0.1.7 I didn't have the error event but I have the crash :(

Maybe it's because now Google Speech have reach v1 ?

[evancohen]

Hey @jaumard thanks for the detailed issue, that crash is concerning.

Were you using the error handler before? I added it to the example in v0.1.8 so it's possible that it just wasn't being reported: https://github.com/evancohen/sonus/commit/466e3357f2db6124400f9f3690b47d447f48270a#diff-553fc2397f3f47643b3a400197616977R23

Have you tried using Sonus on another network? It may be that audio is indeed being streamed too slow. It's also possible that the error is a red haring and it doesn't have to do with the crash. Do you always get the error before it crashes?

It would be super helpful if you could provide some more info here, like a more detailed callstack - this isn't quite enough to go on.

[jaumard]

In fact I don't have anything more on the callstack :( here is all the log I have during the crash Yes I was using the error handler before but this error come from my PR I did, but I don't expect receive it once I have a successful sentence, strange. But sometime I didn't have this error event and crash occurs anyway so maybe not related, I'll try to search a little more details about this crash can be related to the network because for now can't reproduce it... But it append a lot the other day (each time I use it and a hotkey was detected)

[jaumard]

@evancohen I was able to reproduce by simply launch it with no network at all :) then after a long time (I think there timeout is too long) you will see:

{ streamingError: { code: 4, metadata: { _internal_repr: {} } } } as an error event !

And then the crash :

_stream_writable.js:383
  cb();
  ^

TypeError: cb is not a function
    at afterWrite (_stream_writable.js:383:3)
    at onwrite (_stream_writable.js:374:7)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

But sorry nothing more on the call stack than this :/

[evancohen]

Are you saying the hotword and then it eventually crashes, or are you just starting Sonus and it's crashing after a "long time"?

[jaumard]

Yes after the hotword detection ^^ sorry for my test I used directly Sonus.trigger(sonus, 0, "hey lisa"); and then just wait (with no network), after some time I have this crash but everytime not eventually

[evancohen]

Cool, now that I have some repro steps I can try this out. How long should I expect to wait for the crash?

[jaumard]

Few minutes, I'll say 2 or 3 :)

[jaumard]

Any luck for reproducing this problem @evancohen ?

[evancohen]

Sorry, I've been traveling for work so I haven't had the chance - should have some time to look at it tonight though :)

[evancohen]

I was able to reproduce this - it's definitely a network related issue, but it looks to originate from gRPC. A bit more context:

{ Error: Deadline Exceeded
    at ClientDuplexStream._emitStatusIfDone (/Users/evan/Git/sonus/node_modules/grpc/src/node/src/client.js:209:19)
    at ClientDuplexStream._readsDone (/Users/evan/Git/sonus/node_modules/grpc/src/node/src/client.js:177:8)
    at readCallback (/Users/evan/Git/sonus/node_modules/grpc/src/node/src/client.js:237:12) code: 4, metadata: Metadata { _internal_repr: {} } 
}

Unfortunately I don't think there's much we can do from a Sonus perspective here - I'm going to play around with wrapping the streaming event with a try/catch so at least sonus won't crash out when this happens and push a fix if I can get that working reliably.

I would recommend filing an issue on the gRPC repo with the relevant info.

If you want a sonus-free repro you can take a look at: https://gist.github.com/evancohen/e525e9065888f295e3f8e8df5d478d5b

[jaumard]

No problem :) Ho ok cool that you find more informations than me at least :D Is it possible to catch this error and send an error event specific that say 'network problem' or something similar ? At least it doesn't crash the project and we're still able to start again by saying the keyword. That would be perfect until they fix that :)

Do you mind creating the issue as you have more info about it ? And link it here like this I can follow the status please.

If the try/catch doesn't work we should use domain instead to catch this error. I'm fine with sonus thanks :) find it cool and useful. I you don't have too much time I can try to catch the error if you tell me where it's appended because I wasn't able to find it

What do you think ?

[evancohen]

Sure thing - I'll take care of it. I should have some time this weekend to look at it again.
FYI: I'm going on a backpacking trip next week so I'll be off the grid.

[jaumard]

No problem enjoy :) thanks !

[jaumard]

gRPC related issue https://github.com/grpc/grpc/issues/11514

[jaumard]

Just created one for google speech project https://github.com/GoogleCloudPlatform/google-cloud-node/issues/2443

[jaumard]

Can be fix by installing last version of grpc in the project (until the bump the version under google cloud module)

[evancohen]

Woohoo, that's awesome news 🎉 I just got back from the mountains. This may also address #52, which I was never able to reproduce.

Looks like the issue has been patched and all future/re-installations of Google Cloud Speech won't crash.