Int8 pytorch inference

[snehashis1997]

Before Asking

• [X] I have read the README carefully. 我已经仔细阅读了README上的操作指引。

• [X] I want to train my custom dataset, and I have read the tutorials for training your custom data carefully and organize my dataset correctly; (FYI: We recommand you to apply the config files of xx_finetune.py.) 我想训练自定义数据集，我已经仔细阅读了训练自定义数据的教程，以及按照正确的目录结构存放数据集。（FYI: 我们推荐使用xx_finetune.py等配置文件训练自定义数据集。）

• [X] I have pulled the latest code of main branch to run again and the problem still existed. 我已经拉取了主分支上最新的代码，重新运行之后，问题仍不能解决。

Search before asking

• [X] I have searched the YOLOv6 issues and found no similar questions.

Question

Is it possible to inference in int8 as a pytorch model ( as pt file ) ?

Additional

No response

[Chilicyy]

Hi @snehashis1997 , to inference in int8 as pt file may cause large precision decline, and it's common way to inference in int8 as a trt engine file.

[snehashis1997]

Thank you for your reply. if someone wants to export yolov6 fp16 model to openvino fp32 model then using openvino int8 quantization convert the fp320ir model to int8 ir model, can we do that?