Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.46k stars 631 forks source link

[Model Zoo] Potential additions to available catalogue #1191

Open afzalxo opened 1 year ago

afzalxo commented 1 year ago

Hi,

We have been working on FPGA-aware neural architecture search with the objective of maximizing accuracy-throughput tradeoffs of the searched models on Image classification tasks with ImageNet2012. We have come up with a few efficientnet-like models that, in some cases, outperform most high-quality models available in the model zoo in both accuracy and throughput. Here is a list of models that we have found: Target Device DPU Arch Model Name Top-1 Accuracy Float Top-1 Accuracy Quantized E2E throughput (fps), Multi Thread, measured on target
VCK190 DPUCVDX8G_ISA3_C32B6 efficientnet-vck190-a 77.568 76.966 2805.14
VCK190 DPUCVDX8G_ISA3_C32B6 efficientnet-vck190-b 76.656 76.300 4015.04
ZCU102 DPUCZDX8G_ISA1_B4096 efficientnet-zcu102-a 77.698 77.432 271.94
ZCU102 DPUCZDX8G_ISA1_B4096 efficientnet-zcu102-b 76.602 76.314 398.54

The following plots show the comparison in accuracy/throughput against some of the high-quality models currently available in the model zoo. We are comparing against models that take 224 x 224 inputs, and are not pruned, on the ImageNet2012. The numbers for existing models in the plots are obtained from the spreadsheet provided here, while the numbers for our models are measured on the device.

VCK190 Models:

Screenshot 2023-03-03 at 1 56 26 PM

ZCU102 Models:

Screenshot 2023-03-03 at 1 58 10 PM

Would you be interested in adding these models to the model zoo if we can provide the relevant pre-trained models, code, quantized weights, etc?

wangxd-xlnx commented 1 year ago

Hi @afzalxo

Thanks for your comment. Overall I think it's a good idea. We will discuss with marketing in what form it would be better to add your model into model zoo.

Considering the user experience, we hope to be unified in model packaging. It should include float/quantized model, code, and readme.

Could you send us a sample at your convenience? Thanks again for your support.