Use the model on a PC

[ido1990]

Hi! Thank you for the code! Iit seems to be working on the RPi (Altough I don't have a microphone yet but the code runs 🙂). But I was trying to use it on my PC and it doesn't work, I get this error:

  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\tflite_runtime\interpreter.py", line 348, in __init__
    _interpreter_wrapper.CreateWrapperFromFile(
ValueError: Could not open 'wake_word_stop_lite.tflite'.

I removed the GPIO code of course and installed all the required libs. When I try changing the model to one I made on Teachable Machine I get this error:

File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\tflite_runtime\interpreter.py", line 348, in __init__
    _interpreter_wrapper.CreateWrapperFromFile(
SystemError: <built-in method CreateWrapperFromFile of PyCapsule object at 0x0000022BE4B208D0> returned a result with an error set

Do you have any idea how to make it work on a PC? Thanks!

[ShawnHymel]

I have not tried TensorFlow Lite on PC/Windows, so I'm not sure. How did you install it? What version are you running? From what I can tell, the latest version of the tflite runtime is 2.5 (at this time): https://github.com/google-coral/pycoral/releases/. You might also be able to build the wheel manually: https://www.tensorflow.org/lite/guide/build_cmake_pip.

[ido1990]

Thank you for replying. I installed TensorFlow Lite according to the instructions here. The version is 2.5.0 But it seems like something is different...

[ShawnHymel]

Ah! Since making this project, I have learned that TensorFlow Lite versions do not play nicely with each other. I think I generated that model with TFLite converter (either with TFLite v2.0 or 2.1). I foolishly did not write the version down in the project descriptions. You may want to try older versions of TFLite or try re-generating the model with a current version of TensorFlow to see if it will work with TFLite 2.5.

I have a newer project that uses Edge Impulse to generate a wake word model that I highly recommend checking out: https://github.com/ShawnHymel/ei-keyword-spotting. I find that Edge Impulse optimized the MFCCs a lot better than I did, so feature extraction and inference should be much faster. I don't have an example of using a PC for that project, but the Edge Impulse Linux SDK might be a good place to start: https://docs.edgeimpulse.com/docs/linux-python-sdk.

[ido1990]

Ok, I will try re-generating the model again.

I must say that Edge Impulse is such a cool thing! Tried digging in the docs for a few hours but it seems like a complex task to make it work on Windows. The only thing that worked for me so far is compilling the C++ example with my model but it wouldn't be efficient to call it from Python and parse back the output.

UPDATE: I tried loading the file inside Google Colab and it does work (without the sound stream of course). It successfully loaded the original model and also the one I generated on version 2.0.0 Maybe it's something with Windows...

UPDATE 2: I can't even read the H5 I generated in the Colab notebook...

[ShawnHymel]

Yeah, Windows does not play nicely with some of the TensorFlow stuff (such as TensorFlow Lite, from my experience). Something like a Linux Docker image or WSL might be useful here.

[ido1990]

Ok. I managed to get it working just with using the NPZ and H5 files. Then I found another method to load TensorFlow Lite models and it's seems like it's working with a model I trained. Thank you for helping out!

[ido1990]

BTW, the tflite from Teachable Machine still doesn't work. It seems like the shape isn't the same:

[{'dtype': <class 'numpy.float32'>,
  'index': 0,
  'name': 'audio_preproc_input',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
                              'scales': array([], dtype=float32),
                              'zero_points': array([], dtype=int32)},
  'shape': array([    1, 44032]),
  'shape_signature': array([   -1, 44032]),
  'sparsity_parameters': {}}]

[{'dtype': <class 'numpy.float32'>,
  'index': 72,
  'name': 'Identity',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
                              'scales': array([], dtype=float32),
                              'zero_points': array([], dtype=int32)},
  'shape': array([1, 2]),
  'shape_signature': array([-1,  2]),
  'sparsity_parameters': {}}]

When the file produced by your code looks like this:

[{'dtype': <class 'numpy.float32'>,
  'index': 0,
  'name': 'conv2d_input',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
                              'scales': array([], dtype=float32),
                              'zero_points': array([], dtype=int32)},
  'shape': array([ 1, 16, 16,  1]),
  'shape_signature': array([-1, 16, 16,  1]),
  'sparsity_parameters': {}}]

[{'dtype': <class 'numpy.float32'>,
  'index': 21,
  'name': 'Identity',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
                              'scales': array([], dtype=float32),
                              'zero_points': array([], dtype=int32)},
  'shape': array([1, 1]),
  'shape_signature': array([-1,  1]),
  'sparsity_parameters': {}}]

The error is

ValueError: Cannot set tensor: Dimension mismatch. Got 4 but expected 2 for input 0.

When I change the following line to 1, 44032

in_tensor = np.float32(mfccs.reshape(1, mfccs.shape[0], mfccs.shape[1], 1))

I get this error

ValueError: cannot reshape array of size 256 into shape (1,44032)

I tried playing with the settings and values a bit but I think I'm missing something here.

[ShawnHymel]

Glad to hear you got it working! I have not played with Teachable Machine, so I don't know what kind of tensors it expects.

[ido1990]

Glad to hear you got it working! I have not played with Teachable Machine, so I don't know what kind of tensors it expects.

Ok, thank you again!