Open aczhang9 opened 1 year ago
Model on mac accurately detects "Yes," "No," and "Silence." Model is much more sensitive to "Yes" than "No." Also all scores are > 200.
To compile and execute micro_speech example on mac:
git clone git@github.com:robotastic/tensorflow.git
cd tensorflow
make -f tensorflow/lite/experimental/micro/tools/make/Makefile micro_speech
tensorflow/lite/experimental/micro/tools/make/gen/osx_arm64/bin/micro_speech
Model observations:
Heard silence (205) @1696ms
...
Heard yes (202) @4312ms
Heard yes (201) @21768ms
For compilation and execution on the Redboard, I based example code from: https://github.com/robotastic/tensorflow/tree/sparkfun_artemis/tensorflow/lite/experimental/micro/examples/micro_speech
Significant changes in audio_provider.cpp
are needed in order to implement functions that access hardware APIs specific to the Redboard.
Model can correctly detect "Yes" and "No" when they are spoken but during silent periods the model erroneously detects "No" and "Unknown."
Debugging attempted:
audio_provider.cpp
using PDM
class from Apollo3 library: https://github.com/aczhang9/tflite-micro-arduino-examples/blob/2.4.0-ALPHA-precompiled/examples/micro_speech/audio_provider.cppint8
and rest of source code and Arduino library is expecting weights of uint8
. Debugging to do:
Accurately working micro_speech steps:
sparkfun_artemis
branchtensorflow/lite/experimental/micro/tools/make/downloads/
make -f tensorflow/lite/experimental/micro/tools/make/Makefile TARGET=sparkfun_redboard_artemis micro_speech_bin
- Use a jumper wire to connect 3.3V to the right side of the resistor above LED 13.
- While connected, hit the reset button.
- While connected, run the command below:
python3 tensorflow/lite/experimental/micro/tools/make/downloads/AmbiqSuite-Rel2.2.0/boards_sfe/common/tools_sfe/ambiq/ambiq_bin2board.py --bin tensorflow/lite/experimental/micro/tools/make/gen/sparkfun_redboard_artemis_cortex-m4/bin/micro_speech.bin --load-address-blob 0x20000 --magic-num 0xCB -o main_nonsecure_ota --version 0x0 --load-address-wired 0xC000 -i 6 --options 0x1 -b 115200 -port /dev/cu.usbserial-110 -r 2 -v
If python script errors about module/package not found, check python3 --version
and make sure module is installed under the same version pip3 show <package>
.
screen
as a serial monitor screen /dev/tty.usbserial-110 115200
. Observations:
audio_provider.cc
caused an Mbed OS fault when I compiled the example using PlatformIO but the program runs fine under this repo.To do:
micro_speech example contains a model that is trained to recognize verbal "Yes" and "No".