-
## Description
I'm trying to generate a calibration cache file for post-training-quantizatio using Polygraphy.
For which I created custom input json file referring to this [https://github.com/NVIDIA/…
-
## ❓ Question
I have a PTQ model and a QAT model trained with the official pytorch API following the quantization tutorial, and I wish to deploy them on TensorRT for inference. The model is metaforme…
-
I used mtq.INT8_default_CFG as recommended for CNN networks (mtq.quantize(model, config, forward_loop). My initial model ran at 80FPS after quantization it dropped to 40FPS? I checked the model struct…
-
### Search before asking
- [X] I have searched the YOLOv5 [issues](https://github.com/ultralytics/yolov5/issues) and found no similar bug report.
### YOLOv5 Component
_No response_
### Bug
Hi Si…
-
Hello everyone,
Following the Diffusion Models Quanization with Model Optimizer, after this command:
`python quantize.py --model sdxl-turbo --format int8 --batch-size 2 --calib-size 32 --collect-met…
-
## Description
I am using this [calibration script](https://github.com/rmccorm4/tensorrt-utils/tree/master/int8/calibration) to generate the calib cache file for Segformer onnx model. But facing th…
-
## Description
I recently attempted to utilize INT8 quantization with Stable Diffusion XL to enhance inference performance based on the claims made in a recent [TensorRT blog post](https://developer.…
teith updated
6 months ago
-
### System Info
3090 gpu
0.7.1 tensorrt-llm
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppo…
-
## Description
When performing Resnet18 PTQ using TRT-modelopt, I encountered the following issue when compiling the model with TRT.
First off, I started with a pretrained resnet18 from torchvi…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [ Y] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
- [ Y] I am reporti…