-
Hi,
I took out the token embedding layer in Bert and built tensorrt engine to test the inference effect of int8 mode, but found that int8 mode is slower than fp16；
i use nvprof to view the GPU consu…
-
## Description
I'm trying to convert a UNET-like architecutre from ONNX to TensorRT with int8-precision.
Following error is raised after about an hour:
```
[03/18/2023-01:23:30] [TRT] [V] …
-
### 🐛 Describe the bug
I am using
- Nvidia Release 21.08 container
- Which has TensorRT 8.0.1, Pytorch 1.11.0+cu113, Torchvision 0.12.0+cu113, Onnx 1.13.0, Onnxruntime 1.13.1
I have managed to…
-
## Description
## Environment
**TensorRT Version**: 8.6.1
**ONNX-TensorRT Version / Branch**:
**GPU Type**: 3090
**Nvidia Driver Version**: NVIDIA-SMI 535.54.03
**CUDA Version**: 12.2…
-
请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem
- 系统环境/System Environment：Ubuntu18.04 python3.7
- 版本号/Version：Paddle：2.4.1 PaddleOCR：2.6 问题相关组件/Related co…
-
## Description
Say we have an onnx model defined as follows:
```
class MyModel(torch.nn.Module):
def forward(self, x):
x = x + 1
return x
model = MyModel().eval()
dummy =…
-
### Checklist
- [ ] I have searched related issues but cannot get the expected help.
- [ ] 2. I have read the [FAQ documentation](https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/faq.md) but …
-
due to a bug in torch_tensorrt: https://github.com/pytorch/TensorRT/issues/1465#issuecomment-1347630727
One requires 1.3.0, but kaggle has only support for python3.7. This leads to a syntax error.…
-
Hello, I'm trying to do int8 calibration on an ONNX model with C++ API. I see there are samples of INT8 with caffemodel and ONNX MNIST. But how to quantize an ONNX model? Is there any samples or guida…
-
### Checklist
- [X] I have searched related issues but cannot get the expected help.
- [X] 2. I have read the [FAQ documentation](https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/faq.md) but …