search

ricky0123 / vad

Voice activity detector (VAD) for the browser with a simple API
https://www.vad.ricky0123.com
Other
920 stars 146 forks source link

RNNoise #22

Open mkcode opened 1 year ago

mkcode commented 1 year ago

The info in #21 came out me playing quite a bit with trying to make a performant (in iOS Safari) client-side VAD.

I ended up creating a whole new package to get it going: https://github.com/mkcode/vad-js

I originally thought that is was Silero & ONNX was not performant in iOS safari, so I went with using RNNoise. It has a much smaller footprint (~120kb of wasm) than Silero & ONNX and I honestly cannot tell the difference in terms of accuracy.

So I am curious if you had previously evaluated using RNNoise and what you think about it all this.

This is based upon amazing work that I ripped out of jitsi and customized.

There is a working demo that you can check out here: https://github.com/mkcode/vad-js/tree/main/examples/next

ricky0123 commented 1 year ago

Hey @mkcode, wanted to thank you again for looking into this. I didn't know there were issues with iOS Safari. I had come across RNNoise and actually surveyed the jitsi code during my initial research for this project, but I decided at the time that Silero would be easier to implement. If it is equally accurate but smaller then I might eventually try transitioning this project to use it.

I still don't have time to give this the attention it deserves because I am catching up with work after suffering from a bad muscle sprain in my back, but I will eventually look more closely at it (and at your PR).

mkcode commented 1 year ago

Ok. I appreciate the feedback. Looking forward to you returning to this. Please feel free to reach out.

rahulbansal16 commented 3 months ago

Hey @mkcode code, your repo looks interesting. I will try it out for my project

mkcode commented 3 months ago

@rahulbansal16 - Feel free to take a look, but I ended up going with this repo with the changes applied from #21.

I really only brought that repo to a working proof-of-concept level. This repo is much higher quality and I hadn't experienced any issues with the increased CPU load or file download sizes.