search

xiph / rnnoise

Recurrent neural network for audio noise reduction
BSD 3-Clause "New" or "Revised" License
3.89k stars 882 forks source link

Why haven’t you used TensorFlow lite to use your RNNoise model in Android ? Why have you ported into C #185

Open Khubaib96 opened 3 years ago

Khubaib96 commented 3 years ago

Hello Mr. Valin,

Hope you are enjoying the healthy days of your life. I came across RNNoise for noise suppression. I have a query related to your model. • Why haven’t you used TensorFlow lite to use your RNNoise model in Android? • Why have you ported into C?

My basic understanding is that TensorFlow doesn’t support real-time audio-related models with long-term running and efficiency which we can attain from porting model to C. I need your help in clarification of this issue.

ziemek99 commented 3 years ago

I'm not Jean-Marc, but I can already answer the second question: you're right, it's for performance reasons.

All the design and training of the neural network is done in Python using the awesome Keras deep learning library. Since Python is usually not the language of choice for real-time systems, we have to implement the run-time code in C. Fortunately, running a neural network is by far easier than training one, so all we had to do was implement feed-forward and GRU layers. To make it easier to fit the weights in a reasonable footprint, we constrain the magnitude of the weights to +/- 0.5 during training, which makes it easy to store them using 8-bit values. The resulting model fits in just 85 kB (instead of the 340 kB required to store the weights as 32-bit floats).