How to compare the precision layer by layer with engine(for example, tensorRT) if I have a custom operator in my onnx (and a corresponding plugin in TensorRT)? #293
I used to check my engine(tensorRT) layer by layer with onnx using polygraphy tool. But I have a custom operator writen by myself in my onnx, and a corresponding plugin in my tensorRT now and how can I compare them layer by layer using polygraphy?
I write the .sh script like this:
polygraphy run my_model_with_custom_operator.onnx \
--onnxrt --trt \
--trt-outputs mark all \
--onnx-outputs mark all \
--atol 1e-3 --rtol 1e-3 \
--fail-fast \
--val-range [0,1] \
--verbose
And I get error as below:
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: customPlugin(-1) is not a registered function/op
Description
I used to check my engine(tensorRT) layer by layer with onnx using polygraphy tool. But I have a custom operator writen by myself in my onnx, and a corresponding plugin in my tensorRT now and how can I compare them layer by layer using polygraphy?
I write the .sh script like this:
And I get error as below:
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: customPlugin(-1) is not a registered function/opThank you very much.
Environment
TensorRT Version: 8.4.0.6
NVIDIA Driver Version: 535.183.01
CUDA Version: 11.3
CUDNN Version: 8.3.2.44
Operating System:
Python Version (if applicable): 3.9.7
PyTorch Version (if applicable): 1.13.0