search

NVIDIA-AI-IOT / yolo_deepstream

yolo model qat and deploy with deepstream&tensorrt
Apache License 2.0
533 stars 135 forks source link

export onnx error #31

Closed yuanl15 closed 1 year ago

yuanl15 commented 1 year ago

in tensor_quantizer.py line 293

outputs = torch.fake_quantize_per_channel_affine( inputs, scale.data, torch.zeros_like(scale, dtype=torch.int32).data, quant_dim, -bound - 1 if not self._unsigned else 0, bound)

RuntimeError: Zero-point must be Long, found Int

hopef commented 1 year ago

please modify torch.zeros_like(scale, dtype=torch.int32) to torch.zeros_like(scale, dtype=torch.long)

wanghr323 commented 1 year ago

in tensor_quantizer.py line 293

outputs = torch.fake_quantize_per_channel_affine( inputs, scale.data, torch.zeros_like(scale, dtype=torch.int32).data, quant_dim, -bound - 1 if not self._unsigned else 0, bound)

RuntimeError: Zero-point must be Long, found Int

Yes, Agree with @hopef , That is pytorch-quantization 's bug, that should be fixed in the latest version, You can fix this under his reply

I will close this issue, If you have further questions, pls reopen the issue