search

onnx / onnx-tensorrt

ONNX-TensorRT: TensorRT backend for ONNX
Apache License 2.0
2.95k stars 543 forks source link

"Plugin not found, are the plugin name, version, and namespace correct?" Error #800

Open doruksonmez opened 2 years ago

doruksonmez commented 2 years ago

Description

Hi all. I am currently trying to convert Faster RCNN models from TF2 Model Zoo from saved model to TensorRT engine. It is mainly related to Unsupported ONNX data type: UINT8(2) error but I solved it using the following snippet from ELinux: Unsupported ONNX data type: UINT8(2) Fix.

Now I'm having another issue after solving it when I try to convert fixed ONNX model to TensorRT engine using trtexec. I get the following errors:

[E] [TRT] 3: getPluginCreator could not find plugin: Round version: 1
[E] [TRT] ModelImporter.cpp:720: While parsing node number 12 [If -> "StatefulPartitionedCall/Preprocessor/ResizeToRange/cond:0"]:
[E] [TRT] ModelImporter.cpp:721: --- Begin node ---
[E] [TRT] ModelImporter.cpp:722: input: "StatefulPartitionedCall/Preprocessor/ResizeToRange/Less:0"
...
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"`

Environment

TensorRT Version: 8.0.3 ONNX-TensorRT Version / Branch: GPU Type: RTX 3090 Nvidia Driver Version: 470 CUDA Version: 11.3 CUDNN Version: 8.2.4 Operating System + Version: Ubuntu 20.04 Python Version (if applicable): 3.8 TensorFlow + TF2ONNX Version (if applicable): TF 2.6.0 + TF2ONN 1.9.3/1190aa PyTorch Version (if applicable): Baremetal or Container (if container which image + tag): nvcr.io/nvidia/tensorflow:21.10-tf2-py3

Relevant Files

Steps To Reproduce

From SavedModel to ONNX

python3 -m tf2onnx.convert --saved-model faster_rcnn_resnet50_v1_800x1333_coco17_gpu-8/saved_model --output faster_rcnn_resnet50_v1_800x133_trt.onnx --opset 11 --signature_def serving_default --tag serve --target tensorrt

2022-02-03 06:11:51,636 - INFO - Signatures found in model: [serving_default].
2022-02-03 06:11:51,637 - INFO - Output names: ['detection_anchor_indices', 'detection_boxes', 'detection_classes', 'detection_multiclass_scores', 'detection_scores', 'num_detections', 'raw_detection_boxes', 'raw_detection_scores']
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tf2onnx/tf_loader.py:706: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
2022-02-03 06:12:00,428 - WARNING - From /usr/local/lib/python3.8/dist-packages/tf2onnx/tf_loader.py:706: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
2022-02-03 06:12:03,281 - INFO - Using tensorflow=2.6.0, onnx=1.10.2, tf2onnx=1.9.3/1190aa
2022-02-03 06:12:03,281 - INFO - Using opset <onnx, 11>
2022-02-03 06:12:06,739 - INFO - Computed 0 values for constant folding
2022-02-03 06:12:06,742 - INFO - Computed 0 values for constant folding
2022-02-03 06:12:09,388 - INFO - Computed 4 values for constant folding
2022-02-03 06:12:13,661 - INFO - folding node using tf type=ConcatV2, name=StatefulPartitionedCall/GridAnchorGenerator/concat
2022-02-03 06:12:13,662 - INFO - folding node using tf type=Select, name=StatefulPartitionedCall/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/Select_1
2022-02-03 06:12:13,663 - INFO - folding node using tf type=Select, name=StatefulPartitionedCall/SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/Select_1
2022-02-03 06:12:13,663 - INFO - folding node using tf type=Select, name=StatefulPartitionedCall/SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/PadOrClipBoxList/Select_8
2022-02-03 06:12:18,615 - INFO - Optimizing ONNX model
2022-02-03 06:12:40,777 - INFO - After optimization: Add -91 (423->332), BatchNormalization -53 (53->0), Cast -772 (2153->1381), Concat -98 (324->226), Const -4123 (4514->391), Equal -90 (100->10), GlobalAveragePool +1 (0->1), Identity -181 (182->1), Mul -94 (537->443), ReduceMean -2 (2->0), ReduceSum -91 (93->2), Reshape -98 (403->305), Shape -95 (302->207), Slice -13 (431->418), Split -3 (37->34), Squeeze -24 (467->443), Sub -95 (225->130), Transpose -221 (246->25), Unsqueeze -224 (471->247)
2022-02-03 06:12:41,395 - INFO - 
2022-02-03 06:12:41,395 - INFO - Successfully converted TensorFlow model /root/tf2_models/faster_rcnn_resnet50_v1_800x1333_coco17_gpu-8/saved_model to ONNX
2022-02-03 06:12:41,395 - INFO - Model inputs: ['input_tensor']
2022-02-03 06:12:41,395 - INFO - Model outputs: ['detection_anchor_indices', 'detection_boxes', 'detection_classes', 'detection_multiclass_scores', 'detection_scores', 'num_detections', 'raw_detection_boxes', 'raw_detection_scores']
2022-02-03 06:12:41,395 - INFO - ONNX model is saved at /root/tf2_models/faster_rcnn_resnet50_v1_800x1333_trt/faster_rcnn_resnet50_v1_800x133_trt.onnx

From ONNX to TensorRT

trtexec --onnx=/root/tf2_models/faster_rcnn_resnet50_v1_800x1333_trt/faster_rcnn_resnet50_v1_800x133_trt.onnx --fp16 --buildOnly --saveEngine=/root/tf2_models/faster_rcnn_resnet50_v1_800x1333_trt/faster_rcnn_resnet50_v1_800x133_trt.trt

[02/03/2022-06:14:51] [W] [TRT] onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[02/03/2022-06:14:51] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[02/03/2022-06:14:51] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[02/03/2022-06:14:51] [I] [TRT] No importer registered for op: Round. Attempting to import as plugin.
[02/03/2022-06:14:51] [I] [TRT] Searching for plugin: Round, plugin_version: 1, plugin_namespace: 
[02/03/2022-06:14:51] [E] [TRT] 3: getPluginCreator could not find plugin: Round version: 1
[02/03/2022-06:14:51] [E] [TRT] ModelImporter.cpp:720: While parsing node number 12 [If -> "StatefulPartitionedCall/Preprocessor/ResizeToRange/cond:0"]:
[02/03/2022-06:14:51] [E] [TRT] ModelImporter.cpp:721: --- Begin node ---
[02/03/2022-06:14:51] [E] [TRT] ModelImporter.cpp:722: input: "StatefulPartitionedCall/Preprocessor/ResizeToRange/Less:0"
output: "StatefulPartitionedCall/Preprocessor/ResizeToRange/cond:0"
name: "StatefulPartitionedCall/Preprocessor/ResizeToRange/cond"
op_type: "If"
attribute {
  name: "then_branch"
  g {
    node {
      input: "StatefulPartitionedCall/Preprocessor/unstack__88:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/ExpandDims"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Transpose__68:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Transpose__68"
      op_type: "Transpose"
      attribute {
        name: "perm"
        ints: 0
        ints: 3
        ints: 1
        ints: 2
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Transpose__68:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Shape__71:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Shape__71"
      op_type: "Shape"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Shape__71:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_zero__65:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_two__66:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Slice__74:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Slice__74"
      op_type: "Slice"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape"
      op_type: "Shape"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape__54:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape__54"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Shape__54:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:1"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:2"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:3"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack"
      op_type: "Split"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:2"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__57:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__57"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack:1"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__56:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__56"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__57:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_6:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_6"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__56:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_4:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_4"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__57:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_3:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_3"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_2:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_3:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv_1"
      op_type: "Div"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/unstack__56:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_1"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv"
      op_type: "Div"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/truediv_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Min__59:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Min__59"
      op_type: "Min"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Min__59:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_6:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul_1"
      op_type: "Mul"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round_1"
      op_type: "Round"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_7:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_7"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_7:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__62:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__62"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Min__59:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_4:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul"
      op_type: "Mul"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/mul:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round"
      op_type: "Round"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Round:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_5:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_5"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_5:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__61:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__61"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__61:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__62:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Concat__63:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Concat__63"
      op_type: "Concat"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/size_Concat__63:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Cast__75:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Cast__75"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 7
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Slice__74:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Cast__75:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Concat__76:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Concat__76"
      op_type: "Concat"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Transpose__68:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_roi__64:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_empty_float__67:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Concat__76:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Resize__77:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Resize__77"
      op_type: "Resize"
      attribute {
        name: "coordinate_transformation_mode"
        s: "asymmetric"
        type: STRING
      }
      attribute {
        name: "exclude_outside"
        i: 0
        type: INT
      }
      attribute {
        name: "mode"
        s: "linear"
        type: STRING
      }
      attribute {
        name: "nearest_mode"
        s: "floor"
        type: STRING
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Resize__77:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Squeeze:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Squeeze"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Squeeze:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/Identity_graph_outputs_Identity__6:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/ResizeBilinear"
      op_type: "Transpose"
      attribute {
        name: "perm"
        ints: 1
        ints: 2
        ints: 0
        type: INTS
      }
    }
    name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960"
    initializer {
      dims: 1
      data_type: 7
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_zero__65:0"
      raw_data: "\000\000\000\000\000\000\000\000"
    }
    initializer {
      dims: 1
      data_type: 7
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_two__66:0"
      raw_data: "\002\000\000\000\000\000\000\000"
    }
    initializer {
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast_2:0"
      raw_data: "\000\240\246D"
    }
    initializer {
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/resize/Cast:0"
      raw_data: "\000\000HD"
    }
    initializer {
      dims: 0
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_roi__64:0"
      raw_data: ""
    }
    initializer {
      dims: 0
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_const_empty_float__67:0"
      raw_data: ""
    }
    doc_string: "graph for StatefulPartitionedCall/Preprocessor/ResizeToRange/cond then_branch"
    output {
      name: "__inference_Preprocessor_ResizeToRange_cond_true_12975_1960_Preprocessor/ResizeToRange/cond/Identity_graph_outputs_Identity__6:0"
      type {
        tensor_type {
          elem_type: 1
          shape {
            dim {
              dim_param: "unk__9264"
            }
            dim {
              dim_param: "unk__9265"
            }
            dim {
              dim_value: 3
            }
          }
        }
      }
    }
  }
  type: GRAPH
}
attribute {
  name: "else_branch"
  g {
    node {
      input: "StatefulPartitionedCall/Preprocessor/unstack__88:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/ExpandDims"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Transpose__42:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Transpose__42"
      op_type: "Transpose"
      attribute {
        name: "perm"
        ints: 0
        ints: 3
        ints: 1
        ints: 2
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Transpose__42:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Shape__45:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Shape__45"
      op_type: "Shape"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Shape__45:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_zero__39:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_two__40:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Slice__48:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Slice__48"
      op_type: "Slice"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/ExpandDims:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape"
      op_type: "Shape"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape__28:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape__28"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Shape__28:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:1"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:2"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:3"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack"
      op_type: "Split"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:2"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__31:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__31"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack:1"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__30:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__30"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__31:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_6:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_6"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__30:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_4:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_4"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__31:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_3:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_3"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_2:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_3:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv_1"
      op_type: "Div"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/unstack__30:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_1"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 1
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv"
      op_type: "Div"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/truediv_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Min__33:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Min__33"
      op_type: "Min"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Min__33:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_6:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul_1"
      op_type: "Mul"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round_1:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round_1"
      op_type: "Round"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round_1:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_7:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_7"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_7:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__36:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__36"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Min__33:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_4:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul"
      op_type: "Mul"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/mul:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round"
      op_type: "Round"
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Round:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_5:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_5"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 6
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_5:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__35:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__35"
      op_type: "Unsqueeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__35:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Unsqueeze__36:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Concat__37:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Concat__37"
      op_type: "Concat"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/size_Concat__37:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Cast__49:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Cast__49"
      op_type: "Cast"
      attribute {
        name: "to"
        i: 7
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Slice__48:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Cast__49:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Concat__50:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Concat__50"
      op_type: "Concat"
      attribute {
        name: "axis"
        i: 0
        type: INT
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Transpose__42:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_roi__38:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_empty_float__41:0"
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Concat__50:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Resize__51:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Resize__51"
      op_type: "Resize"
      attribute {
        name: "coordinate_transformation_mode"
        s: "asymmetric"
        type: STRING
      }
      attribute {
        name: "exclude_outside"
        i: 0
        type: INT
      }
      attribute {
        name: "mode"
        s: "linear"
        type: STRING
      }
      attribute {
        name: "nearest_mode"
        s: "floor"
        type: STRING
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Resize__51:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Squeeze:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Squeeze"
      op_type: "Squeeze"
      attribute {
        name: "axes"
        ints: 0
        type: INTS
      }
    }
    node {
      input: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Squeeze:0"
      output: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/Identity_graph_outputs_Identity__4:0"
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/ResizeBilinear"
      op_type: "Transpose"
      attribute {
        name: "perm"
        ints: 1
        ints: 2
        ints: 0
        type: INTS
      }
    }
    name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510"
    initializer {
      dims: 1
      data_type: 7
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_zero__39:0"
      raw_data: "\000\000\000\000\000\000\000\000"
    }
    initializer {
      dims: 1
      data_type: 7
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_two__40:0"
      raw_data: "\002\000\000\000\000\000\000\000"
    }
    initializer {
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast_2:0"
      raw_data: "\000\000HD"
    }
    initializer {
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/resize/Cast:0"
      raw_data: "\000\240\246D"
    }
    initializer {
      dims: 0
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_roi__38:0"
      raw_data: ""
    }
    initializer {
      dims: 0
      data_type: 1
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_const_empty_float__41:0"
      raw_data: ""
    }
    doc_string: "graph for StatefulPartitionedCall/Preprocessor/ResizeToRange/cond else_branch"
    output {
      name: "__inference_Preprocessor_ResizeToRange_cond_false_12976_8510_Preprocessor/ResizeToRange/cond/Identity_graph_outputs_Identity__4:0"
      type {
        tensor_type {
          elem_type: 1
          shape {
            dim {
              dim_param: "unk__9266"
            }
            dim {
              dim_param: "unk__9267"
            }
            dim {
              dim_value: 3
            }
          }
        }
      }
    }
  }
  type: GRAPH
}

[02/03/2022-06:14:51] [E] [TRT] ModelImporter.cpp:723: --- End node ---
[02/03/2022-06:14:51] [E] [TRT] ModelImporter.cpp:726: ERROR: builtin_op_importers.cpp:4643 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[02/03/2022-06:14:51] [E] Failed to parse onnx file
[02/03/2022-06:14:51] [I] Finish parsing network model
[02/03/2022-06:14:51] [E] Parsing model failed
[02/03/2022-06:14:51] [E] Engine creation failed
[02/03/2022-06:14:51] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8003] # trtexec --onnx=/root/tf2_models/faster_rcnn_resnet50_v1_800x1333_trt/faster_rcnn_resnet50_v1_800x133_trt_updated.onnx --fp16 --workspace=64 --buildOnly --saveEngine=/root/tf2_models/faster_rcnn_resnet50_v1_800x1333_trt/faster_rcnn_resnet50_v1_800x133_trt.trt
kevinch-nv commented 2 years ago

TensorRT does not support the Round operation yet. TRT will attempt to load user-defined plugins if an operator is not supported, which is why the following error message appears.

[E] [TRT] 3: getPluginCreator could not find plugin: Round version: 1

The operator will be supported in a future release