Open MarcoPrassel opened 8 months ago
Did it work for you ?
@rpatidar No, it did not work. I also tried to take advantage of the Nvidia Deepstream solutions, but the output and input layers have to be changed. I hope @marcoslucianops can help me.
Ohk , I am able to evaluate the standalone model with tensort - https://github.com/lyuwenyu/RT-DETR/blob/main/benchmark/trtinfer.py example shared here
thought tensorrt working should have allowed deepstream to work as well, looks like not working may need some customization.
looking for your expertise @marcoslucianops on how can we use the rt-detr model with the deepstream
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
[NvMultiObjectTracker] Initialized
0:00:00.613182162 48840 0x7feee4002460 WARN nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1174> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
0:00:05.802209521 48840 0x7feee4002460 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1988> [UID = 1]: deserialized trt engine from :/home/pranay/Desktop/model-rt-detr/rtdetr_r101vd_6x_coco_from_paddle.engine
0:00:05.822380989 48840 0x7feee4002460 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2091> [UID = 1]: Use deserialized engine model: /home/pranay/Desktop/model-rt-detr/rtdetr_r101vd_6x_coco_from_paddle.engine
0:00:05.823421857 48840 0x7feee4002460 WARN nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initNonImageInputLayers() <nvdsinfer_context_impl.cpp:1542> [UID = 1]: More than one input layers but custom initialization function not implemented
0:00:05.823429100 48840 0x7feee4002460 ERROR nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1316> [UID = 1]: Failed to initialize non-image input layers
0:00:05.841597013 48840 0x7feee4002460 WARN nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<primary-inference> error: Failed to create NvDsInferContext instance
0:00:05.841751157 48840 0x7feee4002460 WARN nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<primary-inference> error: Config file path: model_config/rtdetr_mode.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 5
0 INPUT kFLOAT images 3x640x640
1 INPUT kINT32 orig_target_sizes 2
2 OUTPUT kFLOAT scores 300
3 OUTPUT kINT32 labels 300
4 OUTPUT kFLOAT boxes 300x4
[NvMultiObjectTracker] De-initialized
I will add the support today. For now, only for this repo: https://github.com/lyuwenyu/RT-DETR
Edit: I'm having issues with the bounding boxes output with the ultralytics model. It's not producing the bbox coordinates when the ONNX model is converted to TensorRT, but it produces the scores.
I can confirms, I am able to use it, haven't tested all batching feature and other stuffs but for single stream its working as expected.
Thanks you so much. I was almost lost on how to get it working.
@marcoslucianops I noticed that there is no benchmark yet. Does it perform well?
I was able to execute the ultralytics based model by adding following patch to export_yolov8.py
- def __init__(self):
+ def __init__(self, img_size):
+ self.img_size = img_size
super().__init__()
def forward(self, x):
- x = x.transpose(1, 2)
+ #x = x.transpose(1, 2)
boxes = x[:, :, :4]
+ boxes[:, :, [0, 2]] *= self.img_size[1]
+ boxes[:, :, [1, 3]] *= self.img_size[0]
scores, classes = torch.max(x[:, :, 4:], 2, keepdim=True)
classes = classes.float()
return boxes, scores, classes
@@ -64,7 +67,7 @@ def main(args):
f.write(name + '\n')
f.close()
- model = nn.Sequential(model, DeepStreamOutput())
+ model = nn.Sequential(model, DeepStreamOutput((640,640)))
Thank you @rpatidar, the multiply to the img_size did the trick.
Hi,
Has the RT-DETR been tested on the Jetson NX? When I try the a trained onnx model on the Jetson NX, the conversion failed with the below message. It works fine on the PC - converted and running. I am using Deepstream 6.0.
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. WARNING: [TRT]: onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped ERROR: [TRT]: ModelImporter.cpp:773: While parsing node number 444 [GridSample -> "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0"]: ERROR: [TRT]: ModelImporter.cpp:774: --- Begin node --- ERROR: [TRT]: ModelImporter.cpp:775: input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_5_output_0" input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_6_output_0" output: "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0" name: "/0/decoder/decoder/layers.0/cross_attn/GridSample" op_type: "GridSample" attribute { name: "align_corners" i: 0 type: INT } attribute { name: "mode" s: "bilinear" type: STRING } attribute { name: "padding_mode" s: "zeros" type: STRING } ERROR: [TRT]: ModelImporter.cpp:776: --- End node --- ERROR: [TRT]: ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter: [8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?" ERROR: Failed to parse onnx file ERROR: failed to build network since parsing model errors.
Hi,
Has the RT-DETR been tested on the Jetson NX? When I try the a trained onnx model on the Jetson NX, the conversion failed with the below message. It works fine on the PC - converted and running. I am using Deepstream 6.0.
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. WARNING: [TRT]: onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped ERROR: [TRT]: ModelImporter.cpp:773: While parsing node number 444 [GridSample -> "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0"]: ERROR: [TRT]: ModelImporter.cpp:774: --- Begin node --- ERROR: [TRT]: ModelImporter.cpp:775: input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_5_output_0" input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_6_output_0" output: "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0" name: "/0/decoder/decoder/layers.0/cross_attn/GridSample" op_type: "GridSample" attribute { name: "align_corners" i: 0 type: INT } attribute { name: "mode" s: "bilinear" type: STRING } attribute { name: "padding_mode" s: "zeros" type: STRING } ERROR: [TRT]: ModelImporter.cpp:776: --- End node --- ERROR: [TRT]: ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter: [8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?" ERROR: Failed to parse onnx file ERROR: failed to build network since parsing model errors.
I have samae question with you
Hi,
Has the RT-DETR been tested on the Jetson NX? When I try the a trained onnx model on the Jetson NX, the conversion failed with the below message. It works fine on the PC - converted and running. I am using Deepstream 6.0.
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. WARNING: [TRT]: onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped ERROR: [TRT]: ModelImporter.cpp:773: While parsing node number 444 [GridSample -> "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0"]: ERROR: [TRT]: ModelImporter.cpp:774: --- Begin node --- ERROR: [TRT]: ModelImporter.cpp:775: input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_5_output_0" input: "/0/decoder/decoder/layers.0/cross_attn/Reshape_6_output_0" output: "/0/decoder/decoder/layers.0/cross_attn/GridSample_output_0" name: "/0/decoder/decoder/layers.0/cross_attn/GridSample" op_type: "GridSample" attribute { name: "align_corners" i: 0 type: INT } attribute { name: "mode" s: "bilinear" type: STRING } attribute { name: "padding_mode" s: "zeros" type: STRING } ERROR: [TRT]: ModelImporter.cpp:776: --- End node --- ERROR: [TRT]: ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter: [8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?" ERROR: Failed to parse onnx file ERROR: failed to build network since parsing model errors.
have you slove this problem?
https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/RTDETR.md
404 - page not found
Hi @marcoslucianops , I am trying to integrate Ultralytics' custom RT-DETR model with Deepstream, but I am having problems with the layer parser from ONNX. Could you please help me ? This is my ONNX's output![image](https://github.com/marcoslucianops/DeepStream-Yolo/assets/109653912/46c3b002-c082-4d80-8163-5d6b2736da55)