Playing on micro-speech

[uraich]

I saw that the new unix build has been merged into main. I tried it and was successful. Now I am trying to understand the micro-speech example. Since you have integrated the audio front end into the microlite module the example code does not work any more. I added: audio_frontend = microlite.audio_frontend() # before audio_frontend.configure() in main.py and micro_speech.py to get that part to work. However I still get an error when trying to create the interpreter:

[mocleiri]

Thanks for this report. The unix port was the first one that ran micro_speech but is now very out of date.

I plan on consolidating the different port scripts into a single script that will work for the different ports. See #50.

Can you try the scripts from the esp32 or rp2 directories stripping out the I2S part?

I'm going to fix this one by using the unix port to run the examples as part of the CI build process.

[uraich]

On the ESP32 version I actually get the same problem:

[mocleiri]

@uraich the model is wrong. Note the UINT8 part. This is an old model that I had from the beginning but it doesn't work. In fact tflm has removed support for UINT8 altogether which is why you get those strange errors.

I filed an upstream documentation fix about this since the model file itself was distributed in a zip file which needed to be updated: https://github.com/tensorflow/tensorflow/issues/48752

I have this working locally and will commit soon.

$ ./micropython main.py
interpreter_make_new: model size = 18712, tensor area = 20496
Process 'No' input of length = 16000

results at 0 = result = -128

results at 1 = result = -119

results at 2 = result = -128

results at 3 = result = 119

maxIndex=3,maxValue=119

Understood No

Process 'Yes' input of length = 16000

results at 0 = result = -128

results at 1 = result = -128

results at 2 = result = 127

results at 3 = result = -128

maxIndex=2,maxValue=127

Understood Yes

The documentation says a score over 200 but that was when the value was 0 to 255. On the esp32 I think I adjust the scoring to add +128 to account for the numbers of int8 that are below zero.

[mocleiri]

I've committed the changes (https://github.com/mocleiri/tensorflow-micropython-examples/pull/58). Can you see if they solve the problem for you?

For ESP32 double check that you are using the model.tflite from the examples/micro-speech/esp32 directory.

[uraich]

I pulled the new repo and tried the unix version, which now works fine for me (same output as shown above) You say, I should use model.tflite from the examples/micro-speech/esp32 directory but there is no model file. I guess that model.tflite from examples/micro-speech/unix should be the version I will have to upload to the ESP32. I don't have the microphone going yet, but the error when creating the interpreter has gone. I will now play on the microphone and as soon as I have that working I will try again.

[mocleiri]

The model file to use is currently here in examples/micro-speech/lib: https://github.com/mocleiri/tensorflow-micropython-examples/blob/main/examples/micro-speech/lib/model.tflite

I will fix this mess in #50 where there will be just one model file and one set of files.