Open YohannXu opened 2 years ago
I can't export the onnx using your script, can you share the onnx model here?
$ python3 2125.py
Traceback (most recent call last):
File "2125.py", line 8, in <module>
torch.onnx.export(model. torch.randn(1, 3, 20, 20, 20), 'conv3d_quant.onnx', opset_version=13)
File "/home/scratch.zeroz_sw/miniconda3/envs/transformer/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1185, in __getattr__
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'QuantConv3d' object has no attribute 'torch'
cc @ttyio
@zerollzeng i fixed code above, you can try it again
could you try adding Q/DQ after the Conv? I am not sure if we have tactics for INT8-input FP32-output 3D convs (will need to check internally).
you mean a mode like this? it doesn't work. @nvpohanh
Thanks for trying. We will debug this. @zerollzeng Could you give it a try with TRT 8.2 and TRT 8.4? Let's see if it's already fixed in TRT 8.4. Thanks
@YohannXu I still met error using your export code:
import torch
from pytorch_quantization import quant_modules
import pytorch_quantization.nn as quant_nn
quant_modules.initialize()
model = nn.Conv3d(3, 16, kernel_size=(3, 3, 3), stride=(1, 1,1), padding=(1, 1, 1))
quant_nn.TensorQuantizer.use_fb_fake_quant = True
torch.onnx.export(model, torch.randn(1, 3, 20, 20, 20), 'conv3d_quant.onnx', opset_version=13)
$ python3 2125.py
Traceback (most recent call last):
File "2125.py", line 6, in <module>
model = nn.Conv3d(3, 16, kernel_size=(3, 3, 3), stride=(1, 1,1), padding=(1, 1, 1))
NameError: name 'nn' is not defined
Looks like they are not modified.
sry, i forgot to import torch.nn, it now works @zerollzeng
I can't reproduce it in TRT 8.4, can you try 8.4?
TensorRT-8.4.1.5/bin/trtexec --onnx=conv3d_quant.onnx --int8
i tried 8.4, but the error still occur. @zerollzeng maybe you can try my onnx model model.zip
I update the library, it now works. Thanks @zerollzeng
Description
I'm using pytorch quantization toolkit to quantize my model, which has some conv3d module. The QAT procedure is OK. But when i use trtexec to convert the onnx with Q/DQ pairs to engine, it failed! I have written some simple code to reproduce the error.
Environment
TensorRT Version: 8.2.5.1 NVIDIA GPU: TITAN RTX NVIDIA Driver Version: 470.103.01 CUDA Version: 11.4 CUDNN Version: Operating System: ubuntu 20.04 Python Version (if applicable): 3.8.10 Tensorflow Version (if applicable): PyTorch Version (if applicable): 1.9.1+cu102 Baremetal or Container (if so, version):
Relevant Files
Steps To Reproduce
use above code to generate a conv3d onnx with q/dq pair, it should look like this.
Then use
it will throw a error
I guess tensorrt doesn't support conv3d onnx with q/dq pair?