search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.87k stars 2.14k forks source link

Can not Convert onnx to tensorrt with calibration on windows with tensorrt8.5.1? #2538

Closed monsterlyg closed 1 year ago

monsterlyg commented 1 year ago

Description

Environment

TensorRT Version: 8.5.1.7 NVIDIA GPU: RTX3080 NVIDIA Driver Version: 512.95 CUDA Version: 11.4 CUDNN Version: 8.5.0 Operating System: win10 Python Version (if applicable): 3.8 Tensorflow Version (if applicable): PyTorch Version (if applicable): Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

I used "trtexec" under Tensorrt8.5.1.7 from here to convert onnx to tensorrt file on windows10 with int8 calibration. It returned error like below:

image

But when i turned to linux, the onnx model could be converted to tensorrt model successfully. I have also tried with Tensorrt8.0.0.3, the same result is it worked on Linux but failed on Windows10. Another phenomenon is that when i dropped "--calib=xxx", it also succeeded on Windows. So there may be something wrong with int8 calibration on windows.

monsterlyg commented 1 year ago

i can provide onnx and calibration file if necessary.

zerollzeng commented 1 year ago

@oxana-nvidia did you see similar issues before? I'm not familiar with the Windows version. Thanks!

oxana-nvidia commented 1 year ago

Hi @monsterlyg, Could you please provide onnx and calibration file so we can test it internally?

for options to try: may be you can consider to increase workspace?

monsterlyg commented 1 year ago

Hi @monsterlyg, Could you please provide onnx and calibration file so we can test it internally?

for options to try: may be you can consider to increase workspace?

I have watched the memory occupied, which is under 4GB.

https://drive.google.com/drive/folders/1gDxg65SaVLwrh3R1bB4HjI7IMPX1J1T9?usp=share_link

oxana-nvidia commented 1 year ago

Thanks for providing data! I've created internal bug 3905480 to track the issue.

monsterlyg commented 1 year ago

I've also tried on windows with 1080Ti and returned success, so the problem may be related with 30xx gpu(just a guess).

oxana-nvidia commented 1 year ago

Hi @monsterlyg, We are not able to reproduce the issue on our side. Could you please try if TensorRT 8.5.2 still has the issue on your side? Also one more suggestion from our engineers was to disable JIT_CONVOLUTIONS tactic source and see if it makes any difference.

oxana-nvidia commented 1 year ago

closing this issue as no updates provided and on our side we can not reproduce it. Please reopen if the issue is still present.