search

emlearn / emlearn-micropython

Efficient Machine Learning engine for MicroPython
83 stars 13 forks source link

Support Audio Classification and Sound Event Detection #6

Open jonnor opened 8 months ago

jonnor commented 8 months ago

Need to at least expose a spectrogram audio feature extraction. There is code for mel spectrogram in emlearn, so that would be the primary candidate https://emlearn.readthedocs.io/en/stable/eml_audio.html

Furthermore would need to have some audio classifier models. The best models here are CNNs and RNNs. At the moment these are not supported in emlearn. But a Multi-Layer-Perceptron (https://github.com/emlearn/emlearn-micropython/issues/1) can also do a decent job. RandomForest can also be used for simple tasks.

Should also have some simple examples. The "simple audio" tutorial from Tensorflow could be a good starting point, https://www.tensorflow.org/tutorials/audio/simple_audio

jonnor commented 1 month ago

There is now an emlearn_fft module included. So that is one piece of the puzzle for a spectrogram implementation.

jonnor commented 1 month ago

The other pieces being a windowing function (like Hann). And a method of reduce dimensionality of FFT a bit (like mel-filters). And a computationally efficient way to do the overlapped windows, before the windowing.