search

espressif / esp-dl

Espressif deep-learning library for AIoT applications
MIT License
516 stars 115 forks source link

Esp-dl VS Tensorflow Lite (AIV-464) #77

Open dankal444 opened 2 years ago

dankal444 commented 2 years ago

Hi, I am currently using Tensorflow Lite models on ESP32 (https://www.tensorflow.org/lite/microcontrollers). I am using mostly conv2d, batchnorm, depth-wise conv2d and relu layers.

Is it worth (and easily achievable) to convert Tensorflow Lite models to esp-dl format. Will there be any performance gain?

If it changes anything: in near future I will be using ESP32-S3

TiramisuJ commented 2 years ago

Hi, We recommend that you use the esp-dl if you want to get the high performance, there are many optimizations for the basic operators in esp-dl. You can refer to this tutorial to convert your model.

dankal444 commented 2 years ago

Thank you TiramisuJ for response.

I have an update. Me and my colegues managed to convert 12 layer model with some random weights to esp-dl and (if we did it correctly as it seems) it is ~10x faster than "vanilla" Tensorflow Lite.

But, we are having big problems in achieving compatibility of results between them. Whole process of conversion is complicated and there are many places we might make some error. Now we struggle to convert even only 2-layer model and have the same results. There is some magic about scale parameter that we do not understand, during quantization it gets generated automatically, with/without sample database.

Would it be possible in future to have a tool that just takes tflite model, and outputs all those 3 .cpp files (one could call them layers.cpp, layers.hpp and model.cpp) that esp-dl uses? We have possibility to use: -tflite float32 model -tflite fully converted to int8 model -> this should be fairly easy to convert to esp-dl int8 model right?

ESP-YHY commented 2 years ago

Hi @dankal444, Thank you for your good advice. So far, we provide a quantization tool kit to help produce the exponent parameter. It can help convert from an ONNX model to ESP-DL format coefficients. We have a plan to produce a model too in the future.