PTQ和QAT生成的文件如何转换成.engine文件

ModelTC / MQBench

Model Quantization Benchmark

Apache License 2.0

766 stars 140 forks source link

Closed Shining-Tears closed 2 years ago

Shining-Tears commented 2 years ago

现在可以通过PTQ/QAT生三个文件，包括两个onnx和一个json，那么该如何变成tensorrt的文件格式呢？

tuochi commented 2 years ago

例子代码库有链接

Shining-Tears commented 2 years ago

谢谢~ 例子中的onnx生成的engine是fp32的吧，在后面加上--clip-range-file <jison文件位置>能生成int8的engine吗？

Shining-Tears commented 2 years ago

然后试了一下，两个engine文件大小并没有差距很大，正常情况int8的engine不是应该比fp32的engine小很多吗？

Shining-Tears commented 2 years ago

@tuochi 麻烦了~

github-actions[bot] commented 2 years ago

This issue has not received any updates in 120 days. Please reply to this issue if this still unresolved!