search

analogdevicesinc / ai8x-synthesis

Quantization and Synthesis (Device Specific Code Generation) for ADI's MAX78000 and MAX78002 Edge AI Devices
Apache License 2.0
55 stars 47 forks source link

Synthesis of models trained with Pytorch Quantization API #275

Closed tasptz closed 1 year ago

tasptz commented 1 year ago

What would be the best way to synthesize models obtained with quantization aware training via pytorch quantization api?

MaximGorkem commented 1 year ago

PyTorch QAT can be either static or dynamic, quantizing the weights and/or the activations. The scale factor is learned for the quantized parameters. However, this scale factor is a floating number which makes PyTorch training unusable directly since MAX78000/2 works with integer arithmetic. The scale factors should be power of two exponent to be represented as a bit-shift in the integer operations. PyTorch team seems to be working on that but to the best of our knowledge, there is no stable support for it yet.

Please check the discussion: https://discuss.pytorch.org/t/how-to-set-quantization-aware-training-scaling-factors/65872/7

Instead, we recommend our QAT tool, which is designed considering the limitations of MAX78000/2. https://github.com/MaximIntegratedAI/ai8x-training#quantization-aware-training-qat