evancohen / sonus

:speech_balloon: /so.nus/ STT (speech to text) for Node with offline hotword detection
MIT License
625 stars 79 forks source link

v0.1.9 #67

Closed evancohen closed 6 years ago

evancohen commented 6 years ago

@jaumard does this address your concern?

jaumard commented 6 years ago

Look like it! Did you push a npm version of this? I might be able to test later today :)

evancohen commented 6 years ago

I haven't yet because my build is failing now. I'll push another pre-release later today and let you know when it's ready for testing :)

evancohen commented 6 years ago

@jaumard I published 0.1.9-2 with the next tag for you to test.

jaumard commented 6 years ago

Thanks! I'll try to test asap and let you know :) today will be difficult but tomorrow should be fine

jaumard commented 6 years ago

So currently testing, look much more stable in case or errors, but I still have a problem sometime that is just stay in 'listen' state indefinitely (or very long time) and I have to trigger the hot word again and say something to "unlock" the state. But this is more the issue of the timeout.

evancohen commented 6 years ago

Back from vacation :) Glad this is a step in the right direction, but I'd like to actually understand how listening can get into a bad state. There's some edge case not being handled correctly.

I also have to fix these build errors (which seem to be caused by dependencies). I should be able to fix them in the next few days.

jaumard commented 6 years ago

Hey @evancohen :) hope your vacation were good ! I use this version since few days now and it's really a big improvement ! I don't have to restart my project when there an error now so that's a big step ;)

Here is what I did on my project, I just plug my leds to sonus event, when hotword event is fired, I turned leds green, in case of error in red (and off after some time), and when I have the final-result I turn them off. And in some cases they just stay green so this mean that final-result is never fired.

For now can't say more than this about this other issue :/

evancohen commented 6 years ago

I think the edge case is that StreamingRecognizeResponse doesn't always send a response.

What's unclear to me is if we can always expect a END_OF_SINGLE_UTTERANCE for the SpeechEventType when single_utterance is set to true. This is what we depend on today.

Kinda coding out loud here, but maybe we shouldn't be looking at the SpeechEventType at all...

killStream = () => {
    cloudSpeechRecognizer.listening = false
    audioStream.unpipe(recognitionStream)
    recognitionStream.end()
}

.streamingRecognize(request)
.on('error', error => {
    // Emit error
    killStream()
})
.on('data', data => {
        if(data.results[0] && data.results[0].alternatives[0]) {
        // Emit Partial or final transcript
        // If final:
            killStream()
        } else {
        // Reached transcription time limit
        killStream()
        }
    )
);
evancohen commented 6 years ago

Ok, I think those changes should address the root issue (finally). I did make an incompatible API change and added a timeout event to replace the hack for empty final results.

In the interest of backwards compatibility I think I'll just publish a minor release that sends an empty final results instead of the timeout, but for the next update (with more cloud recognizers) I'll move to the new model.

evancohen commented 6 years ago

@jaumard I've done some testing on this and it seems very stable :) This is much cleaner and should gracefully handle the case you described above (with no final transcript). I published 1 final prerelease sonus@0.1.9-4, if you could give that a test pass that would be awesome.

jaumard commented 6 years ago

Ho you've been busy :D I'll setup this new version this evening and test it for few days and let you know how it goes by the end of week.

jaumard commented 6 years ago

Everything setup on two different device, let's see during the next days how it goes ;)

jaumard commented 6 years ago

So far so good @evancohen ! Great improvements :) I still test it but I think you can publish this one, I'll open new issue or reopen if I have the problem again as it append randomly maybe I just get lucky lol But for errors look like it's 100% fixed

sdetweil commented 6 years ago

when I try to get the latest, I get

pi@raspberrypi:~/smart-mirror $ npm install sonus@0.1.9 smart-mirror@0.0.15 /home/pi/smart-mirror └── (empty)

never mind.. had to upgrade npm to latest for this install to work