search

analogdevicesinc / ai8x-synthesis

Quantization and Synthesis (Device Specific Code Generation) for ADI's MAX78000 and MAX78002 Edge AI Devices
Apache License 2.0
55 stars 49 forks source link

Preparing a pre-trained Keras model for deployment #168

Closed asti205 closed 2 years ago

asti205 commented 2 years ago

Currently I am trying to deploy a CNN model onto MAX78000 that has been trained using Keras (*.h5). I converted the model to ONNX and extracted the architectural description to YAML. I then tried generating the C files using ai8xize.py, however I am getting errors that my YAML file contains unknown keys. Comparing the YAML file of my model (generated from original Keras model) and some of the YAML sample files in the folder ai8x-synthesis/networks, the architecture description looks very different. How can I obtain the architecture description as expected by ai8xize from my Keras model?

It would be great if I do not have to retrain the CNN model using PyTorch to ensure comparability towards other platforms, where the toolchain is Keras-based.

Another question: As TensorFlow >=2.6.0 is not supporting YAML anymor, but only JSON: Will there be an adaptation in the toolchain?

Kind regards, asti205

Jake-Carter commented 2 years ago

Hello,

The YAML file used by our synthesis tool is a custom format specific to the tool. It's not related to TensorFlow's YAML file. Some manual effort will be necessary to bring over your model description into our format.

There is a YAML Quickstart guide that you can reference.

However, re-training is always required for our hardware. More details can be found in the readme, but the model needs to be retrained for hardware rounding and clipping. With smaller models targeted at embedded devices re-training is typically not a huge roadblock.

I would recommend porting your model over towards PyTorch so you can use our training tools. You'll want to ensure that your model is compatible with our hardware (See "Limitations of MAX78000 Networks") and use our custom PyTorch classes (see Custom nn.modules). This is an in-depth subject, so I would recommend reviewing the entire readme.

In short: you'll need to port over to our tools and retrain since we don't support Keras. As you do so, you'll want to make sure your model is compatible with our hardware.

asti205 commented 2 years ago

Hi Jake, so I assume no further steps towards the integration of pretrained Keras & TFLite models without using the Maxim-specific toolchain are planned?

rotx-eva commented 2 years ago

That's correct. Pretrained models would not work as intended since they don't "know" about clipping/rounding/quantization that happens in hardware. However, with the size of the networks that are typically used on MAX78000, retraining is achievable in a reasonable amount of time using a reasonable amount of GPU resources.