leaphy-robotics / leaphy-blocks

The web-based visual programming editor.
Apache License 2.0
1 stars 0 forks source link

[research] Can we use tinyML on RP2040 to detect sounds? #101

Open rmoesbergen opened 5 months ago

rmoesbergen commented 5 months ago

Some RP2040 based board have a built-in microphone and have the ability to run tinyML. There are examples on the internet of using this to train a tiny ML model that can detect specific sounds (like hand clapping, siren, whistle, dog bark, etc.) This would be a really cool block to have to allow a leaphy robot to "listen" to its creator.

rmoesbergen commented 5 months ago

https://blog.tensorflow.org/2021/09/TinyML-Audio-for-everyone.html

https://colab.research.google.com/github/ArmDeveloperEcosystem/ml-audio-classifier-example-for-pico/blob/main/ml_audio_classifier_example_for_pico.ipynb#scrollTo=Ojuc2yoIrA8G

https://github.com/karolpiczak/ESC-50