search

emlearn / emlearn-micropython

Efficient Machine Learning engine for MicroPython
51 stars 13 forks source link

Support Convolutional Neural Networks #7

Open jonnor opened 2 months ago

jonnor commented 2 months ago

It would be great to support some basic CNNs. In a manner that is just as easy to install and use as the tree-based-ensembles that we have now. That means that there should be a core module that supports CNN models, but that the particular model/weights can be loaded at runtime.

CNNs would enable many computer vision tasks, such as Image Classification and Object Detection. But they are also frequently used for audio tasks, using spectrograms as the input. Thus it is also relevant part of or complement to #6

jonnor commented 2 months ago

TensorFlow Lite for Microcontrollers is one of the most established here. But I find it to be quite large - just the interpreter takes 16 kB+. And I believe that it being in C++ may make it extra challenging to support as a dynamic native module (the mpy-ld linker is quite limited). It is also already available in OpenMV, so those who want to use it with MicroPython can already access it from there.

jonnor commented 2 months ago

I have evaluated and tested (in C) two frameworks with support for CNNs: https://github.com/majianjia/nnom and https://github.com/sipeed/TinyMaix/

Of these, only TinyMaix provides a loadable model format. For NNoM one would have to create one's own format, which is a pain, and would require additional tooling both on the host side and the MicroPython side. I also like the simplicity of TinyMaix.

jonnor commented 1 month ago

Initial module code merged in #8

Need still to test on device and provide some documentation / examples.