pytorch / TensorRT

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
https://pytorch.org/TensorRT
BSD 3-Clause "New" or "Revised" License
2.58k stars 350 forks source link

🐛 [Bug] Error compiling when you use TensorRT INT8 calibrator (implemented using native TRT python API) with Torch-TRT #1196

Closed peri044 closed 1 year ago

peri044 commented 2 years ago

Bug Description

If a tensorrt calibrator implemented using TRT python API is passed to Torch-TRT compilation, it fails due to the object not being recognized.

Error :

======================================================================
ERROR: test_compile_script (__main__.TestAccuracy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_ptq_trt_calibrator.py", line 118, in test_compile_script
    trt_mod = torchtrt.ts.compile(self.model, **compile_spec)
  File "/home/dperi/Downloads/new_env/lib/python3.8/site-packages/torch_tensorrt/ts/_compiler.py", line 113, in compile
    compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
  File "/home/dperi/Downloads/new_env/lib/python3.8/site-packages/torch_tensorrt/ts/_compile_spec.py", line 181, in _parse_compile_spec
    info.ptq_calibrator = compile_spec["calibrator"]
TypeError: (): incompatible function arguments. The following argument types are supported:
    1. (self: torch_tensorrt._C.ts.CompileSpec, arg0: torch_tensorrt._C.IInt8Calibrator) -> None

Invoked with: <torch_tensorrt._C.ts.CompileSpec object at 0x7f7fc65f0b70>, <__main__.TRTEntropyCalibrator object at 0x7f7fc0ca5040>

To Reproduce

Run https://github.com/pytorch/TensorRT/blob/master/tests/py/test_ptq_trt_calibrator.py

Expected behavior

https://github.com/pytorch/TensorRT/blob/master/tests/py/test_ptq_trt_calibrator.py should pass

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

Any version of Torch-TRT (regardless of environment) should be able to work

github-actions[bot] commented 2 years ago

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

github-actions[bot] commented 1 year ago

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days