aczhang9 / redboard-tflite

0 stars 0 forks source link

Compile and execute micro_speech example on Redboard #1

Open aczhang9 opened 1 year ago

aczhang9 commented 1 year ago

micro_speech example contains a model that is trained to recognize verbal "Yes" and "No".

aczhang9 commented 1 year ago

Model on mac accurately detects "Yes," "No," and "Silence." Model is much more sensitive to "Yes" than "No." Also all scores are > 200.

To compile and execute micro_speech example on mac:

  1. git clone git@github.com:robotastic/tensorflow.git
  2. compile: (source code in https://github.com/robotastic/tensorflow/tree/master/tensorflow/lite/experimental/micro/examples/micro_speech)
    cd tensorflow
    make -f tensorflow/lite/experimental/micro/tools/make/Makefile micro_speech
  3. execute:
    tensorflow/lite/experimental/micro/tools/make/gen/osx_arm64/bin/micro_speech

Model observations:

...

Heard yes (202) @4312ms

Heard yes (201) @21768ms

aczhang9 commented 1 year ago

For compilation and execution on the Redboard, I based example code from: https://github.com/robotastic/tensorflow/tree/sparkfun_artemis/tensorflow/lite/experimental/micro/examples/micro_speech

Significant changes in audio_provider.cpp are needed in order to implement functions that access hardware APIs specific to the Redboard.

Model can correctly detect "Yes" and "No" when they are spoken but during silent periods the model erroneously detects "No" and "Unknown."

aczhang9 commented 1 year ago

Debugging attempted:

Debugging to do:

aczhang9 commented 1 year ago

Accurately working micro_speech steps:

  1. clone this repo: git@github.com:robotastic/tensorflow.git and checkout sparkfun_artemis branch
  2. manually download https://developer.arm.com/-/media/Files/downloads/gnu-rm/7-2018q2/gcc-arm-none-eabi-7-2018-q2-update-mac.tar.bz2 to tensorflow/lite/experimental/micro/tools/make/downloads/
  3. Compile: make -f tensorflow/lite/experimental/micro/tools/make/Makefile TARGET=sparkfun_redboard_artemis micro_speech_bin
  4. Upload binary to board:
    - Use a jumper wire to connect 3.3V to the right side of the resistor above LED 13.
    - While connected, hit the reset button.
    - While connected, run the command below: 
    python3 tensorflow/lite/experimental/micro/tools/make/downloads/AmbiqSuite-Rel2.2.0/boards_sfe/common/tools_sfe/ambiq/ambiq_bin2board.py --bin tensorflow/lite/experimental/micro/tools/make/gen/sparkfun_redboard_artemis_cortex-m4/bin/micro_speech.bin --load-address-blob 0x20000 --magic-num 0xCB -o main_nonsecure_ota --version 0x0 --load-address-wired 0xC000 -i 6 --options 0x1 -b 115200 -port /dev/cu.usbserial-110 -r 2 -v

    If python script errors about module/package not found, check python3 --version and make sure module is installed under the same version pip3 show <package>.

  5. Use screen as a serial monitor screen /dev/tty.usbserial-110 115200.

Observations:

To do: