reco stops working, smartword detected, grey band at bottom of screen never goes away.

[sdetweil]

Expected Behavior

don't know how to reproduce.. say 'smart mirror', see mirror display gray band, and then nothing reco works again.. gray band does not go away.. all other functions working.. clock, calendar events update, sleep works, motion wakeup works..

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

1. turn on autosleep 2.use go to sleep command
2. say smart mirror..
3. screen wakes, but reco no longer works.

Context

mirror no longer working, have to recycle it. remote reload gets one smart word detection, then hung again.

Your Environment

• Environment name and version (e.g. Raspberry Pi 3, node.js 5.4): Pi3 Raspian, SM code cloned 2/17/17
• Operating System and version (e.g. Raspbian, Ubuntu, OSX):

[evancohen]

I saw this in an older version. This could also be cause by an internet connection issue. I'll take a look this week.

[sdetweil]

"version": "0.0.10",

note the recreate with autosleep, voice wakeup my PI is hard wired.. not wireless

[sdetweil]

I added a console.log("!f:", "") (what sonus sends after reco complete, but with no text) after the Focus.change("default"); in the autosleep service.wake() function.. to 'finish' the voice reco function. this cleans up and allows reco to work again. (altho the !f shows on the console log

[sdetweil]

hm.. doesn't work for one scenario.. autosleep timeout sleep, voice wakeup.. reco is toast after that

[sdetweil]

this might be related to #553 I updated sonus to 0.1.5 even tho I just did a new install, (manual, npm install) it built new modules.

now smart-mirror without a follow-on word times out. we will see..

[evancohen]

I suggest upgrading to the latest version of Sonus (0.1.7). I should have some time to look at this tomorrow

[sdetweil]

i just upgraded Sonus to 0.1.7 (JUST did this the other day, and got 0.1.5?) and so far this doesn't happen. will watch thruout the day

no luck, altho partially not as bad.. have grey bar, reco does not work.. but, put to sleep via remote web ui, THEN 'smart mirror' causes wake up, but reco doesn't work grey bar still present

[evancohen]

It's just a UI bug then. The fix is probably to remove the white bar if no results come in within ~5 seconds. Patched the sonus version in dev with f9f78d64d6921629356de061ef4de73c606ac1d5

[sdetweil]

yeh, but it appears sonus is out of sync with the UI..

[sdetweil]

I added console logging to the speech service event handlers.. like this ipcRenderer.on('hotword', () => { console.log("hotword detected"); <--- callbacks.listening(true) }) ipcRenderer.on('partial-results', (event, text) => { console.log("partial detected ",text); callbacks.partialResult(text) })

        ipcRenderer.on('final-results', (event, text) => {
    console.log("final detected ",text);
            callbacks.finalResult(text)
            annyang.trigger(text)
            callbacks.listening(false)
        })

when we got 'out of synch', i always get a hotword detected event, but never anything else. so we are not getting google speech service events from the sonus process.

also, on my system the process is still using rec instead of arecord event tho I have sonus 01.7 installed

[sdetweil]

I added some debugging.. which shows the recognizer getting started, but then nothing coming from it (and once its hung like this it never recovers) I added this to module/sonus.js (can't post JS files here) // When a hotword is detected pipe the audio stream to speech detection detector.on('hotword', (index, hotword) => { sonus.trigger(index, hotword) sonus.emit('partial-result',"detector trigger") })

csr.on('error', error => sonus.emit('error', { streamingError: error }))

let transcriptEmpty = true csr.on('data', data => { const result = data.results[0] if (result) { transcriptEmpty = false if (result.isFinal) { sonus.emit('final-result', result.transcript) Sonus.annyang.trigger(result.transcript) transcriptEmpty = true //reset transcript } else { sonus.emit('partial-result', result.transcript) } } else if (data.endpointerType === 'END_OF_UTTERANCE' && transcriptEmpty) { sonus.emit('final-result', "") } else { sonus.emit('partial-result', "not clean result") } })

sonus.trigger = (index, hotword) => { if (sonus.started) { try { let triggerHotword = (index == 0) ? hotword : models.lookup(index) sonus.emit('hotword', index, triggerHotword) CloudSpeechRecognizer.startStreaming(opts, sonus.mic, csr) sonus.emit('partial-result', "recognizer streaming") } catch (e) { sonus.emit('partial-result',{ streamingError: ERROR.INVALID_INDEX }) throw ERROR.INVALID_INDEX } } else { throw ERROR.NOT_STARTED } }

return sonus }

here is the log of the dev window. near the end I did smart-mirror, show map.. show map did not appear, stuck on grey bar on screen

motion start detected

speech hotword detected speech partial detected recognizer streaming <--- after CloudSpeechRecognizer.startStreaming

speech partial detected detector trigger

motion end detected

---- no completion of speech detection

say smart mirror again, get sonus responding, ---- new smart-mirror speech hotword detected ---- no recognizer started.... is prior hung?

-1488224888725.log.txt

the autosleep service.wake function,(called on motion start) wrapped in a check for sleeping, else is a do nothing service.wake = function (actual) { if(Focus.get() === 'sleep'){ service.woke = true; if (config.autoTimer.mode == "monitor") { service.exec(config.autoTimer.wakeCmd, service.puts); } else if (config.autoTimer.mode == "tv") { // is this a real wakeup, or the fake one to handle the enerystar power off problem if(actual == true){ // if the timer was running if(EneryStarTimer !=null){ // stop it $interval.cancel(EneryStarTimer) EneryStarTimer = null; } // if the dummy wake up delay is running, stop it too if(EneryStarTimerStop !=null){ $interval.cancel(EneryStarTimerStop) } } } Focus.change("default"); SpeechService.cleanup(); } };

[sdetweil]

I switched to use arecord instead of rec... same problem.. after some time, (with sleep/wake cycles and motion events) we get the hotword recognized, the streaming setup, but then nothing, ever again.. altho we do still get the hotword detection..

how can I debug the google speech streaming?

[justbill2020]

it seems you need to rebase your repo you're working on... google speech streaming has zero debug as its still in beta technically... something is off on your build... i'm using the dev branch with commit f9f78d64d6921629356de061ef4de73c606ac1d5 and it works fine no issues...

Even when I use the master branch with commit 031dca5b0d4632ad3b993fb5556536ae2d240e28 and i cannot reproduce this...

i have tried this both on the pi 3 and in ubuntu

[sdetweil]

if you use the web ui to put it to sleep and wake up, then use voice to sleep and wake up, then voice to sleep and motion to wake up... then wait 20 minutes, then say 'smart mirror', wait a few seconds, then show map.. you will not get a response.. , and never again.. I added debug (see above), and get the recognizer started without error, but it never generates any events anymore.. as this (sonus) is an external process, it will have no knowledge of what is going on with SM.. so it should work the same way all the time..

[sdetweil]

ok, I have added debugging to the 3 modules (modules/sonus/index.js, sonus.js and plugins/speech/service.js, all attached) to capture what is going on... they write to a common log file.

so far, the log (attached) shows the same results as before.. get hotword, setup for streaming, then nothing.. this log records ALL the coded events.. and tries to keep them in time order.

here is a sample

module detector silence
module detector silence
module detector sound
module detector sound
module detector sound
module detector sound
module detector sound

module detector trigger hotword
    task signal hotword
        speech hotword detected
module detector trigger streaming ready
module detector ready for more
module detector sound
module detector silence
module detector silence
module detector silence
module detector silence
module detector silence
module detector silence
module detector sound
module detector sound
module detector csr noise detected
module detector sound
module detector sound
module detector csr partial show
    task signal partial show
        speech partial detected show

module detector sound
module detector csr partial show me
    task signal partial show me
        speech partial detected show me

module detector sound
module detector csr partial show map
    task signal partial show map
        speech partial detected show map

module detector sound
module detector silence
module detector silence
module detector silence
module detector silence
module detector silence
module detector silence
module detector csr noise detected
module detector csr noise detected
module detector csr final result show map
    task signal final show map
        speech final detected show map

---

here is the start of the failure, search for 'final' til no more found, then search for 'hotword' from there at this point the system has been idle with no voice for a few minutes.. the log is not timestamped.. but I am considering adding that too..

module detector silence
module detector sound
module detector sound
module detector sound
module detector sound
module detector sound

module detector trigger hotword
    task signal hotword
        speech hotword detected
module detector trigger streaming ready
module detector ready for more
module detector sound
module detector silence
module detector silence

now need to work on the google code for logging..

sonus.log.txt sonus.js.txt service.js.txt index.js.txt

[justbill2020]

still unable to reproduce this...

[sdetweil]

do you have motion enabled too?

I thought it might be pipe related motion and sonus tasks in the background together, but the lkog clearly shows the sonus task not getting and generating events..

100% repeatable here.. in less than 5 minutes.

[justbill2020]

yes i followed your steps to follow... i do not have your external motion in my build i'm using the most recent commit of the dev and master branches on the evancohen repo... works fine there

[justbill2020]

i would assume there's a possible utilization issue on the capture

[sdetweil]

I 'think' it is related to motion..

[justbill2020]

correct... we're saying the same thing... i think its related to the process you're using to capture images to detect motion... i think it either has a mem leak or is using too much CPU causing your issue...

[sdetweil]

i hear ya.. the timestamped logs with CPU performance monitor along side say different.. will post a little later.. want to see if I can instrument the google libs too..

[evancohen]

@sdetweil can you npm ls sonus? I wonder if the version that you have on your system is old - new versions should always fire an event when recognition is complete (ie: when the connection is broken, regardless if it gets results or not)

[sdetweil]

0.1.7 just installed to make sure. Failed on 0.1.5 too

[evancohen]

There's a chance that it's a network issue - it's possible that grpc or the speech library is silently killing the connection at some point. I just got a replacement Pi so I can try to reproduce this again over the weekend.

[sdetweil]

i understand.. but we should catch that... otherwise the darned thing ends up being useless...

I am on the hunt..

[evancohen]

The only way to catch it of it's ending silently is to specify a timeout on our end. For instance if you detect a hotword and don't get results for 5 seconds we remove the glow.

Super simple fix for a potentially intricate problem. My favorite :)

[sdetweil]

but THAT is a symptom of the problem, but NOT the problem.. (I added that reset the screen.. it doesn't help.. the speech module is toast)..

the detector no longer works..

here is the failure, with a little more detail heard sound, detected hotword, setup for more, good setup hears sound, but we don't get speech events ever again.. we detect hotword repeatedly.. but speech reco is dead..

anymore (part of attached log)..

3/2/2017 12:47:42 PM module detector silence
3/2/2017 12:47:42 PM module detector silence
3/2/2017 12:47:42 PM module detector sound
3/2/2017 12:47:42 PM module detector sound
3/2/2017 12:47:42 PM module detector sound
3/2/2017 12:47:43 PM task motion started
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM    autosleep motion started
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:43 PM module detector trigger hotword
3/2/2017 12:47:43 PM    task signal hotword
3/2/2017 12:47:43 PM module detector trigger streaming ready
3/2/2017 12:47:43 PM module detector ready for more   **<----- waiting for speech events which never come.**
3/2/2017 12:47:43 PM        speech hotword detected  
3/2/2017 12:47:43 PM module detector sound
3/2/2017 12:47:44 PM module detector sound
3/2/2017 12:47:44 PM module detector sound
3/2/2017 12:47:44 PM module detector sound
3/2/2017 12:47:44 PM module detector silence
3/2/2017 12:47:44 PM module detector silence
3/2/2017 12:47:44 PM module detector silence
3/2/2017 12:47:44 PM module detector silence
3/2/2017 12:47:44 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector sound
3/2/2017 12:47:45 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:46 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:47 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:48 PM module detector silence
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:49 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:50 PM module detector sound
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:51 PM module detector silence
3/2/2017 12:47:52 PM module detector silence
3/2/2017 12:47:52 PM module detector sound
3/2/2017 12:47:52 PM module detector sound
3/2/2017 12:47:52 PM module detector sound
3/2/2017 12:47:52 PM module detector sound
3/2/2017 12:47:52 PM module detector sound
3/2/2017 12:47:52 PM module detector silence
3/2/2017 12:47:52 PM module detector silence
3/2/2017 12:47:53 PM module detector silence
3/2/2017 12:47:53 PM module detector silence
3/2/2017 12:47:53 PM module detector silence
3/2/2017 12:47:53 PM module detector silence
3/2/2017 12:47:53 PM module detector silence

ps.log.txt sonus.log.txt

[evancohen]

Have you tried running sonus and the motion script(s) directly to see if any other errors bubble up?

[sdetweil]

yes, and no issues.. without the SM code running..

tried again., and can get the failure with just the two npm run tasks going.. no 'other' errors.. reco just stops eventing

[sdetweil]

I rebooted and ran just the npm run sonus, no motion daemon, no npm motion,

and I get the exact same failure.. bring it up and let it sit.. do a reco, then hotword, let it time out.. sit for at least half an hour, then it does hotword reco, but nothing else

debug shows same as above.. after streaming setup, 'sound' is heard, but no reco events.

I have done an npm update, and grpc reinstall/rebuild. and checked for platform updates. and when I installed the latest, it was to a clean folder. the old one was renamed out of the way

[sdetweil]

so, I think we need this anyhow.. if the reco engine dies (bug, network, whatever), we need to restart it if it cannot self recover

how do I start an interval timer in main.js?
start a timer on receipt of hotword detected, if no final in 10 (whatever) seconds, then restart the background process.

[sdetweil]

well, some progress. in modules/sonus/index.js is this code

CloudSpeechRecognizer.startStreaming = (options, audioStream, cloudSpeechRecognizer) => {
  if (cloudSpeechRecognizer.listening) {
    return
  }

when the recognizer fails, it is left in listening mode.

if I comment out this return, then smart-mirror recovers, until the next time.. (startStreaming cleans up open pipes)

but the net of this sounds like the stream setup failed for some reason..

we got this

3/5/2017 9:31:39 AM module detector trigger hotword
3/5/2017 9:31:39 AM     task signal hotword
3/5/2017 9:31:39 AM module detector trigger streaming ready
3/5/2017 9:31:39 AM module detector ready for more
3/5/2017 9:31:39 AM         speech hotword detected
3/5/2017 9:31:39 AM module detector sound
3/5/2017 9:31:39 AM module detector sound
3/5/2017 9:31:39 AM module detector silence
3/5/2017 9:31:39 AM module detector silence
3/5/2017 9:31:39 AM module detector silence
3/5/2017 9:31:39 AM module detector silence
3/5/2017 9:31:40 AM module detector silence
3/5/2017 9:31:40 AM module detector silence
3/5/2017 9:31:40 AM module detector silence
3/5/2017 9:31:40 AM module detector silence
3/5/2017 9:31:40 AM module detector sound
3/5/2017 9:31:40 AM module detector csr noise detected
3/5/2017 9:31:40 AM module detector sound
3/5/2017 9:31:40 AM module detector sound
3/5/2017 9:31:40 AM module detector csr partial show
3/5/2017 9:31:40 AM     task signal partial show
3/5/2017 9:31:40 AM         speech partial detected show

3/5/2017 9:31:40 AM module detector sound
3/5/2017 9:31:41 AM module detector csr partial show me
3/5/2017 9:31:41 AM     task signal partial show me
3/5/2017 9:31:41 AM         speech partial detected show me

3/5/2017 9:31:41 AM module detector sound
3/5/2017 9:31:41 AM module detector csr partial show map
3/5/2017 9:31:41 AM     task signal partial show map
3/5/2017 9:31:41 AM         speech partial detected show map

3/5/2017 9:31:41 AM module detector sound
3/5/2017 9:31:41 AM module detector sound
3/5/2017 9:31:41 AM module detector silence
3/5/2017 9:31:41 AM module detector silence
3/5/2017 9:31:41 AM module detector silence
3/5/2017 9:31:41 AM module detector silence
3/5/2017 9:31:41 AM module detector silence
3/5/2017 9:31:42 AM module detector silence
3/5/2017 9:31:42 AM module detector silence
3/5/2017 9:31:42 AM module detector silence
3/5/2017 9:31:42 AM module detector silence
3/5/2017 9:31:42 AM module detector csr noise detected
3/5/2017 9:31:42 AM module detector csr no longer listening
3/5/2017 9:31:42 AM module detector csr noise detected
3/5/2017 9:31:42 AM module detector csr final result show map
3/5/2017 9:31:42 AM     task signal final show map
3/5/2017 9:31:42 AM         speech final detected show map
---- done
3/5/2017 9:37:01 AM module detector sound
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector trigger hotword
3/5/2017 9:37:02 AM     task signal hotword
3/5/2017 9:37:02 AM module detector trigger streaming ready
3/5/2017 9:37:02 AM module detector ready for more
3/5/2017 9:37:02 AM         speech hotword detected
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector sound
3/5/2017 9:37:02 AM module detector silence
3/5/2017 9:37:03 AM module detector silence
3/5/2017 9:37:03 AM module detector silence
3/5/2017 9:37:03 AM module detector silence
3/5/2017 9:37:03 AM module detector silence
3/5/2017 9:37:03 AM module detector silence
3/5/2017 9:37:03 AM module detector sound
3/5/2017 9:37:03 AM module detector sound
3/5/2017 9:37:03 AM module detector sound
3/5/2017 9:37:04 AM module detector sound
3/5/2017 9:37:04 AM module detector sound
3/5/2017 9:37:04 AM module detector sound
3/5/2017 9:37:04 AM module detector sound
3/5/2017 9:37:04 AM module detector silence
3/5/2017 9:37:04 AM module detector silence
3/5/2017 9:37:04 AM module detector silence
------ nothing
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:44 AM module detector sound
3/5/2017 9:37:45 AM module detector trigger hotword
3/5/2017 9:37:45 AM     task signal hotword
3/5/2017 9:37:45 AM already listening, new request aborted <----- (return)
3/5/2017 9:37:45 AM module detector trigger streaming ready
3/5/2017 9:37:45 AM module detector ready for more
3/5/2017 9:37:45 AM         speech hotword detected
3/5/2017 9:37:45 AM module detector sound

======
======
comment out return
3/5/2017 9:46:31 AM module detector sound
3/5/2017 9:46:31 AM module detector sound
3/5/2017 9:46:31 AM module detector sound
3/5/2017 9:46:31 AM module detector sound
3/5/2017 9:46:32 AM module detector sound
3/5/2017 9:46:32 AM module detector trigger hotword
3/5/2017 9:46:32 AM     task signal hotword
3/5/2017 9:46:32 AM already listening, new request aborted  <---- already listening return commented out
3/5/2017 9:46:32 AM module detector trigger streaming ready
3/5/2017 9:46:32 AM module detector ready for more
3/5/2017 9:46:32 AM         speech hotword detected
3/5/2017 9:46:32 AM module detector sound
3/5/2017 9:46:32 AM module detector sound
3/5/2017 9:46:32 AM module detector sound
3/5/2017 9:46:32 AM module detector silence
3/5/2017 9:46:32 AM module detector silence
3/5/2017 9:46:32 AM module detector silence
3/5/2017 9:46:33 AM module detector silence
3/5/2017 9:46:33 AM module detector silence
3/5/2017 9:46:33 AM module detector silence
3/5/2017 9:46:33 AM module detector sound
3/5/2017 9:46:33 AM module detector sound
3/5/2017 9:46:33 AM module detector csr noise detected
3/5/2017 9:46:33 AM module detector sound
3/5/2017 9:46:33 AM module detector sound
3/5/2017 9:46:33 AM module detector csr partial show  <-----   recovered, working again
3/5/2017 9:46:33 AM     task signal partial show
3/5/2017 9:46:33 AM         speech partial detected show

3/5/2017 9:46:33 AM module detector sound
3/5/2017 9:46:33 AM module detector csr partial show me
3/5/2017 9:46:33 AM     task signal partial show me
3/5/2017 9:46:33 AM         speech partial detected show me

3/5/2017 9:46:34 AM module detector sound
3/5/2017 9:46:34 AM module detector sound
3/5/2017 9:46:34 AM module detector csr partial show map
3/5/2017 9:46:34 AM     task signal partial show map
3/5/2017 9:46:34 AM         speech partial detected show map

3/5/2017 9:46:34 AM module detector sound
3/5/2017 9:46:34 AM module detector silence
3/5/2017 9:46:34 AM         wait for reco timeout
3/5/2017 9:46:34 AM module detector silence
3/5/2017 9:46:34 AM module detector silence
3/5/2017 9:46:34 AM module detector silence
3/5/2017 9:46:34 AM module detector silence
3/5/2017 9:46:35 AM module detector silence
3/5/2017 9:46:35 AM module detector silence
3/5/2017 9:46:35 AM module detector silence
3/5/2017 9:46:35 AM module detector csr noise detected
3/5/2017 9:46:35 AM module detector csr no longer listening
3/5/2017 9:46:35 AM module detector csr noise detected
3/5/2017 9:46:35 AM module detector silence
3/5/2017 9:46:35 AM module detector csr final result show map
3/5/2017 9:46:35 AM     task signal final show map
3/5/2017 9:46:35 AM         speech final detected show map

[sdetweil]

got this error today reco completely unresponsive

3/7/2017 11:36:13 AM module detector trigger hotword 3/7/2017 11:36:13 AM task signal hotword 3/7/2017 11:36:13 AM module detector trigger streaming ready 3/7/2017 11:36:13 AM module detector ready for more 3/7/2017 11:36:13 AM speech hotword detected 3/7/2017 11:36:13 AM module detector sound 3/7/2017 11:36:13 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector sound 3/7/2017 11:36:14 AM module detector trigger hotword 3/7/2017 11:36:14 AM task signal hotword 3/7/2017 11:36:14 AM already listening, new request aborted 3/7/2017 11:36:14 AM speech hotword detected 3/7/2017 11:36:14 AM module detector trigger start streaming error=ReferenceError: recognitionStream is not defined 3/7/2017 11:36:23 AM wait for reco timeout

[DannyLuu]

@sdetweil I am getting this error too. I will try to take a look into this error this weekend.

[sdetweil]

I can send u my files with trace enabled.. sonus.js, modules/sonus/index.js, and plugins/speech/index.js, they will generate the above log file info

[sdetweil]

@DannyLuu any luck with debug of this?

[nevido]

@sdetweil did u resolve your issue.. ? because i've been have same problem.. for last weekend i tried to resolve it.. but failed. T T i don't know the reason

[sdetweil]

@nevido No.. @DannyLuu was assigned to look a this problem .. I have been unable to debug it.

[nevido]

@sdetweil how about use annyang instead of google speech ?

[justbill2020]

@nevido,

we do use annyang... from my understanding annyang relies on the browser's speech recognition ability (which is why you can click "speak!" on the remoteUI and have it work.) That's also why you can't use an IOS device and click "speak!" because its forbidden in IOS to access the microphone directly. however on a mac/linux/windows pc you can use chrome or firefox and the "speak!" button works.

the entire mirror app is within electron which is a chromium based app engine to create apps with nodejs, html, css and js so that you don't have to HTTP server, I think electron's website says it best "Electron uses Chromium and Node.js so you can build your app with HTML, CSS, and JavaScript."

Electron uses/used Chromium speech API to allow for voice recognition... the issue is google restricted the API to 50 calls per day... When this happened we had to require users to get their own Chromium Speech API keys which kinda sorta skirted the Chromium Speech API TOS... but since electron is a chromium project and we were using it for that... it was ok...

When google cloud speech API was released we redev'd how voice works in this project and it increased performance! Not sure what's changed since then... other than possibly the underlying sonus dependency... in the dev branch we did stop using rec and began using arecord... however that only increased performance...

@evancohen has stated that he would look at this this weekend... so hopefully we move forward on this soon.

[nevido]

@justbill2020 thank you for ur help.. thank u

[sdetweil]

do we have any status on this issue?

[sdetweil]

that status list seems a little wonky.. in progress AND on hold?

how can I help move the ball?

[DannyLuu]

@sdetweil I haven't had time to take a look into this.

Do you have an idea of what is causing this? I feel like the previous merge with the majority of changes contains clues as to why this bug is happening. If we can trace down what was committed and confirm that it was the cause it would be a lot easier to fix.

[sdetweil]

I do not.. it is 'around' the setup of the stream to the google speech api..

[sdetweil]

Ok, here is the log from npm run sonus

sonus.log.txt three hotword detections..

1st - show map 2nd, no response after hotword 3rd - crash

[justbill2020]

is this now resolved... it seems so based on conversations on discord

[sdetweil]

new pull seems to have resolved