-
## Question
I would like to understand more on how TRT under the hood converts Q/DQ models to use actual INT8 kernels. Specifically, how are the scale factors being propagated. Specifically I wonde…
-
### System Info
Linux
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported task in the `examp…
-
Does TensorRT support QAT&PTQ INT8 quantization of clip/vit models? Could you please provide any relevant quantization examples and accuracy & latency benchmark?
shhn1 updated
10 months ago
-
It would be awesome if the repo supported quantization methods.
Reference: [k-quants](https://github.com/ggerganov/llama.cpp/pull/1684)
-
Hey,
How to do quantization to int8 model of img_stage_lt_d.onnx and bev_stage_lt_d.onnx?
Thanks!
-
Hello, is it possible to use the camera directly in the model to generate results in real time? Like yolo, you can input the camera data and get the result in real time.
-
Right now ao works just fine to quantize an arbitrary HF model
However this simple workflow is failing meaning we don't really interop well with the rest of the HF ecosystem
```python
from tr…
-
Hi,
It is pytorch resnet50 predefined fp32 model
I have taken aimet_resnet50_int8 onnx model (sim.export api) and converted into dlc and validating dlc model with imagenet dataset on target.
but i…
-
大佬,我按照文档执行了第10步导出模型,执行成功了,然后导出了一堆文件,原始模型用的是qwen2-1.5B-Instruct,modelscope下的,然后按照第十二步:模型的量化方法可以在文件夹 "Do_Quantize" 中查看。我改成我自己的路径,但是总是报各种各样的错误,能针对 原始模型到onnx模型 或者 原始模型到org格式模型出个详细点的教程吗?我用的mac
-
### Branch/Tag/Commit
main
### Docker Image Version
nvcr.io/nvidia/pytorch:23.04-py3
### GPU name
T4
### CUDA Driver
470.141.03
### Reproduced Steps
I'm trying to run `calib…