search

ModelTC / MQBench

Model Quantization Benchmark
Apache License 2.0
762 stars 137 forks source link

PTQ和QAT生成的文件如何转换成.engine文件 #120

Closed Shining-Tears closed 2 years ago

Shining-Tears commented 2 years ago

现在可以通过PTQ/QAT生三个文件，包括两个onnx和一个json，那么该如何变成tensorrt的文件格式呢？

image
tuochi commented 2 years ago

例子代码库有链接

Shining-Tears commented 2 years ago

谢谢~ 例子中的onnx生成的engine是fp32的吧 ，在后面加上--clip-range-file <jison文件位置>能生成int8的engine吗？

Shining-Tears commented 2 years ago

然后试了一下，两个engine文件大小并没有差距很大，正常情况int8的engine不是应该比fp32的engine小很多吗？

Shining-Tears commented 2 years ago

@tuochi 麻烦了~

github-actions[bot] commented 2 years ago

This issue has not received any updates in 120 days. Please reply to this issue if this still unresolved!