NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.84k stars 2.14k forks source link

DCNV2 plugin tensorrt 8.5 arm int8 convert error #2698

Closed mairkiss closed 1 year ago

mairkiss commented 1 year ago

hello, i asked the same question before,i am using tensorrt8.5 to convert my onnx model to int8 model, the special thing is that my onnx model contains a custom DCNV2 plugin, The python implementation of DCNV2 is based on the following library: https://github.com/CharlesShang/DCNv2/blob/master/dcn_v2.py

But when converting the model, I always get such an error: ....... [2023-01-31 06:14:12 WARNING] Missing scale and zero-point for tensor 728, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [2023-01-31 06:14:12 WARNING] Detected invalid timing cache, setup a local cache instead [2023-01-31 06:14:19 WARNING] No implementation of layer Split_120[SplitPlugin] obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation. [2023-01-31 06:14:20 ERROR] 2: [pluginV2Runner.cpp::getInputHostScale::88] Error Code 2: Internal Error (Assertion scales.size() == 1 failed.) terminate called after throwing an instance of 'std::runtime_error' what(): Failed to create object

The previous suggestion was to update the version of tensorrt to 8.5, so I used the latest version below to replace: 图片 I download two 8.5 versions, x86 and arm, x86 can convert success, but arm failed, I don't know if there is a problem with my arm version download?i use cuda 11.4, platform: orin driver

zerollzeng commented 1 year ago

Looks like your model is building with INT8 but your plugin doesn't have the INT8 scale.

zerollzeng commented 1 year ago

Can you provide a reproduce for this error? I can help take a check if I got spare time.

ttyio commented 1 year ago

closing since no activity for more than 3 weeks. thanks!