search

evancohen / smart-mirror

The fairest of them all. A DIY voice controlled smart mirror with IoT integration.
http://smart-mirror.io
2.8k stars 694 forks source link

Speech commands not working with my usb mic #553

Closed tgeci closed 7 years ago

tgeci commented 7 years ago

I am trying to get speech command running, but they are not working for my setup. There is also a problem with the detection language of the speech.

Expected Behavior

App should react on Hotword or other audio commands. Also it should recognize german words, if de-DE is configured in config.json.

Current Behavior

No reaction. I checked my mic setup. Recording with my pi User and arecord command works fine and I can play it on my mac without problem.

Possible Solution

I think thats there is a matter with the annyang locale, but did not find where. I

Steps to Reproduce (for bugs)

No reaction on microphone inputs. Only way to get voice recognition is using the Smart Mirror Remote. If I use Button "Speak" on Smart Mirror Remote the mirror app interprets the german word "Hallo" as the english word "hello".

Your Environment

tgeci commented 7 years ago

While troubleshooting this matter I came across sonus.js. I created a Google Computed Account and saved & configured the JSON Keyfile, but I think it will not be executed or loaded in my setup. Inserted some console.log debug messages in sonus js file to verify this and there was no output of them. Whats the usecase of this library/plugin ? We have annyjang for voice recognition, for what we need sonus ? Is it needed for hotword detection in combination with snow ?

Could you give me a hint, how I can disable hotword detection to locate the error ?

Greetings Taulant

justbill2020 commented 7 years ago

Does rec t.wav Work? Did you rftm and follow it to the letter?

evancohen commented 7 years ago

I was not able to reproduce this error. Setting the language to de-DE was able to recognize my pretty horrible German.

Some background Annyang doesn't do any of the speech recognition anymore because it no longer works in Electron (it only does command parsing): https://github.com/TalAter/annyang/issues/174#issuecomment-268864762

Sonus does the hotword detection and streaming recognition. Also, it doesn't use arecord it uses sox to get 16-bit signed-integer linear pulse modulation encoded audio.

Some suggestions/solutions

tgeci commented 7 years ago

npm run sonus didn't work for me because some dependencies with some C libs, which should be builded. Finally I got it running with a new clean raspbian installation. Language recognition works fine with localisation. It seems to be slow and feels like hanging the first 2-3 commands are ok, then it hangs. I will try to troubleshoot and change the mic intensity.

Is there any way to avoid hotword detection ? Would be nice using instead of Hotword Motion Detection. For example: If motion detected -> listen for commands. 2 Min without motion, turn off.

DannyLuu commented 7 years ago

Hey @tgeci, I get the same hanging when issuing the hot-word.

By any chance are you using the Raspberry Pi 2B or lower? If so I think it is just a hardware issue.

justbill2020 commented 7 years ago

@tgeci can you provide the error so we can help you... saying some error doesn't help us help you... the more info the better

justbill2020 commented 7 years ago

Also if Sonus won't work and voice recognition uses sonus motion detection instead of the hotword won't help

tgeci commented 7 years ago

@DannyLuu I am using Raspberry PI 3.

@justbill2020 Yes, I followed the manual step by step. I coudn't access the PI today, because I wasn't at home. I will try to reproduce the sonus dependency errors tomorrow and attach here some output. Whereby I dont need to run the npm run sonus command, because the mic is working after new install like mentioned in my last comment. Even so I will try it out to exclude possible dependency errors.

justbill2020 commented 7 years ago

I'm sorry I didn't see it was resolved... 2-3 seconds between hotword detection and listening is normal...

so if this is resolved should we close out the issue?

evancohen commented 7 years ago

I'm guessing that the dependencies that you are missing are these: https://docs.smart-mirror.io/docs/install_dependencies.html

2-3 seconds is an awfully long delay... Once the mirror glows you should be able to start speaking. That said, there may be a ~1 second lag between saying "smart mirror" and the mirror glowing, that's to be expected.

tgeci commented 7 years ago

@evancohen I don't think, that this dependencies are missing. I ran the quick installer script: https://raw.githubusercontent.com/evancohen/smart-mirror/master/scripts/pi-install.sh. I will check it out this evening

@justbill2020 I check, if there is still any dependency problem and related to that I will update oder close this issue.

tgeci commented 7 years ago

There are no more dependency problems. I think there were related to the OS. Before I did the clean install, I had an older raspbian release (pre Pixel Desktop).

I close this issue, thanks for support!

evancohen commented 7 years ago

I took another look at this and found out that we get a HUGE performance boost using arecord over rec. I've pushed that in a PR here: #564

@tgeci can you test the latest on the dev branch and let me know what you think?

tgeci commented 7 years ago

@evancohen Oh cool! Yes, I will test it this evening.

evancohen commented 7 years ago

Forgot to mention: you'll also need to update sonus to its latest version.

npm install sonus@0.1.5
tgeci commented 7 years ago

@evancohen I can confirm the performance enhancement. Voice recognition works now really smooth & fine. Commands are getting recognized much better.

Good Job!