-
Hi, can you share best practices for quantization for CNN models?
Are the modelopt quantized PTQ is the way to go with tensorrt for cnn models (resnet retinanet etc)? I was able to quantize retinanet…
-
Hi Ryan,
I am trying to create a calibration file for the ResNet-18 Caffe model. You have mentioned the below statement in another issue:
_I have created a reference for INT8 calibration on Ima…
-
### System Info
- GPU Name: T4 X2
- System Ram: 30GB
### Who can help?
_No response_
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Reproducti…
-
## Description
I am trying to figure out if TensoRT and the `pytorch_quantization` module support post-training quantization for vision transformers.
The following piece of code follows the `pyt…
-
Which model do you want to support in mmdeploy? Comment here. You can like a comment to increase model priority.
| | Model | Codebase | Backend | Number of likes | Released
| :--- |…
-
Example "detectnet.py --model=facedetect "images/humans_*.jpg" images/test/facedetect_humans_%i.jpg" doesnt work. In this case, I tried to run tao-converter that downloaded together with *.etlt model,…
-
我按照文档要求已经按照了包含trt的paddle,
paddlepaddle_gpu-2.2.1-cp37-cp37m-linux_x86_64.whl
在设置参数--run_mode=trt_int8 --trt_calib_mode=True跑picodet导出模型的的infer.py时候报错
File "/home/vehicle_detection/PaddleDetecti…
-
I used the self-trained llama13b model in tensorrt-llm 0.7.1 environment, using 2 4090 gpu, running multiple batches of the dataset, generating abnormal results. However, when running a single batch i…
-
**Describe the solution you'd like**
I found that the latest release of tensorrt 8.0 is support for the int8 quantization on GPU, which is great accelerate inference speed.
And now onnxruntime is …
-
## Description
## Environment
**TensorRT Version**:8.6
**NVIDIA GPU**:A10
**NVIDIA Driver Version**:525.147.05
**CUDA Version**:12.0
**CUDNN Version**:8.9
Operating Sy…