search

evancohen / sonus

:speech_balloon: /so.nus/ STT (speech to text) for Node with offline hotword detection
MIT License
625 stars 79 forks source link

Using Sonus in Electron #15

Closed evancohen closed 7 years ago

evancohen commented 7 years ago

In the interest of documenting this issue:

There are a number of issues using native node modules in electron. Since Sonus depends on Snowboy and Google Cloud Speech, both of which have native components, it's difficult to get Sonus running in an Electron project.

Typically the solution would be to use electron-rebuild but this seems to fail for gRPC (and take a substantial amount of time on a low powered device for snowboy).

Existing Issues:

What's left to try? Creating a node child process (not a pretty solution, but should work by running the processes outside of the Node runtime). This doesn't work well (read as "at all") for installable Electron apps...

NetOpWibby commented 7 years ago

Hmm, damn.

I went to the annyang repo to search for Electron issues and then I found your project. I didn't know Electron had difficulty with native node modules. Unfortunate, but I hope that changes soon.

While I'm here, I'd like to say that I like this project and will be following development of it. I love the idea of decentralized voice recognition and was going to start developing today, but seeing as how I was intending to use Electron, I'll have to postpone.

My question is this...is is possible to prevent the sending of voice data to Google? I read some time ago that they store voice data on their servers for who knows how long and I'd like my privacy-minded users to feel at ease when using my software.

evancohen commented 7 years ago

I've found a work-around to allow the use of gRPC in Electron for my smart-mirror project. I ended up spawning a child process and using an IPC (which let's you use whatever node env you have on your machine, regardless of Electron version). Depending on how you are distributing your Electron package it may work for you.

I can almost guarantee that Google stores your voice data indefinitely (albeit anonymously). I really want to start integrating alternate recognition services soon, but we're currently facing some audio issues that are a bigger concern (#22).

If you're interested in adding support for a cloud recognizer that you think would work your your privacy-minded users I'd be happy to point you in the right direction.

NetOpWibby commented 7 years ago

Ah yes, I've used IPC for my browser project. Huh, I never thought about utilizing it for custom node modules but it makes sense!

I would most certainly love to know of an alternative to Google's offerings!

jeremytripp commented 7 years ago

Hi @evancohen, I'm getting a "Failed at the grpc@0.15.0 install script 'node-pre-gyp install --fallback-to-build'" error when I try to install Sonus as part of a new Magic Mirror module I'm building. Would that be related to Electron issue you mentioned above? If so, can you expand upon your workaround? Thanks -- excited to test out Sonus!

evancohen commented 7 years ago

@jeremytripp Electron doesn't play well with prebuilt node binaries (because it packages it's own version of V8). If you take a look at my smart mirror you should be able to figure it out. Long story short you can spawn a child process and use an IPC to get data from Sonus. Happy to help if you run into any issues :)

As an asside: we're always looking for contributors for the smart mirror!

jerocosio commented 7 years ago

Is there some place where I can see how to easily add Sonus to Electron? I allready tried copying what you have done with the smart mirror, but still no luck I'm getting the same error as everyone else:

npm ERR! grpc@0.15.0 install: node-pre-gyp install --fallback-to-build npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the grpc@0.15.0 install script 'node-pre-gyp install --fallback-to-build'.

evancohen commented 7 years ago

As of today, the npm module grpc@1.1.0-pre1 should now work on electron. Make sure that you follow the instructions in the electron docs for installing native modules. You should be able to omit the line 

export npm_config_build_from_source=true

to install the precompiled binary file for electron.

I haven't had the chance to try this, but I'll update this thread with some instructions once I get it working.

evancohen commented 7 years ago

A small update here: it seems that a change introduced into snowboy now requires some slight modification to get things working. Details here: Kitt-AI/snowboy#126

evancohen commented 7 years ago

A late night and some head scratching has produced a working version of sonus in electron: https://github.com/evancohen/sonus-electron-boilerplate

I'm going to consider this issue resolved!

NetOpWibby commented 7 years ago

Awesome! This is great news.

rhclayto commented 7 years ago

On Electron 1.7.5, Raspberry Pi 3, trying to install grpc (1.6.0), it doesn't find any pre-built grpc binaries for that combination, even though grpc is now supposedly making binaries available for node on the ARM architecture. I opened an issue for this at https://github.com/grpc/grpc/issues/12535 .

I was however able to rebuild grpc with electron-rebuild without errors by implementing the fix to grpc at https://github.com/grpc/grpc/issues/12536 .

Just noting this here for future searchers.