search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.43k stars 621 forks source link

Pytorch QAT #1188

Open ash8327 opened 1 year ago

ash8327 commented 1 year ago

Hi,

I am trying to use Pytorch's native QAT instead of pytorch_nndct and then use Vitis AI's Quantization and Compilation for a VCK190. Is there a way to do this? If not would the new ONNX compatibility enable a QAT fp32 Pytorch model to be run on the ONNX runtime in Vitis AI 3.0?

The reason I am exploring this is we need to run both on GPU and FPGA devices with QAT trained models without having to modify the model for pytorch_nndct.

Alexiazzf commented 1 year ago

Hi @ash8327 ,

I am sorry that there is no way to do this yet. The quantization should correspondence to hardware but Pytorch's native QAT cannot do it. There are some different quantization strategy between Vitis AI's Quantization and Pytorch's native QAT.