search

MycroftAI / mycroft-precise

A lightweight, simple-to-use, RNN wake word listener
Apache License 2.0
851 stars 229 forks source link

Tensorflow Lite #66

Open george-ai opened 5 years ago

george-ai commented 5 years ago

Are there any future plans to use tensorflow lite?

MatthewScholefield commented 5 years ago

Tensorflow lite doesn't currently directly support lstm layers. It sounds like it might be possible by manually implementing it, but we won't be looking into it until we've found an example of someone else doing this successfully online.

goddade commented 5 years ago

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/examples/lstm/g3doc Can this help?

andreselizondo-adestech commented 4 years ago

Do we have any news on this? It'd be great to have support for this on precise-runner (for lightweight applications). Training wouldn't need to be done on TF Lite.

andreselizondo-adestech commented 4 years ago

Hello all! I've forked the project and updated it to use TF 2.2.0 (currently using 2.2.0rc2 as official release is yet to come) and Keras 2.3.1 You can find the fork here: https://github.com/andreselizondo-adestech/mycroft-precise

A few things to note:
- Removed all pocketsphynx dependencies and scripts.
- TF 2.x doesn't expose graphs/sessions (only if using tf.compat.v1). This fork makes things work without explicitly sharing sessions between TF and Keras.
- TF 2.1 doesn't support GRU layers correctly for tflite. TF 2.2 is minimum requirement.
- The only scripts I tested for compatibility are the ones I use:
  - precise-train (train keras model)
  - precise-convert (convert keras model to tflite)
  - precise-test (test keras or tflite models)
  - precise-listen (listen on keras or tflite models)
- Added a TFLiteRunner that allows using tflite models with the previous scripts.
- Added my custom training parameters for my model on scripts/train.py
- I couldn't get TF 2.x to support TensorBoard. Decided it was not important.

I'm no expert on TF/Keras usage, so I might've butchered some things. However, I believe this is a good starting point for resolving this issue. Let me know if I can be of further assistance!

krisgesling commented 4 years ago

Eltocino just flagged this TFlite accelerator library as a possibility: https://blog.tensorflow.org/2020/07/accelerating-tensorflow-lite-xnnpack-integration.html

Posting it here so I don't forget it, and in case anyone's interested in trying it out.