how can i use my own ONNX model with ros_deep_learning package ?

[kleanbotmk2]

Hello,

I would like to know how I can use my own ONNX model that I trained thanks to your tutorials in the ros_deep_learning package!

For the moment I'm using this line which works but uses the default model: "ros2 launch ros_deep_learning video_viewer.ros2.launch input:=/dev/video0 output:=display://0"

And also do you know how can i change the image resolution, the framerate etc.. ?

Thanks for your help,

Sacha

[dusty-nv]

@kleanbotmk2 the various ROS topic params that you can set are found here: https://github.com/dusty-nv/ros_deep_learning#topics--parameters

Regarding which kind of ONNX models you can use, please refer to the Hello AI World tutorial which this is built on: https://github.com/dusty-nv/jetson-inference#hello-ai-world

[kleanbotmk2]

So that means that I cannot change the image resolution if I use the ros_deep_learning package ?

And I tried to use my onnx model with the right parameter but it's not working however it's working with the normal Jetson inference package (the detectnet command)! Can you give the right command line to execute ?

[kleanbotmk2]

any ideas ?

[dusty-nv]

So that means that I cannot change the image resolution if I use the ros_deep_learning package ?

@kleanbotmk2 as I linked to above, there are ROS params exposed that allow you to set the dimensions of the input stream: https://github.com/dusty-nv/ros_deep_learning/tree/master#video_source-node

Regarding your ONNX detection model, are you setting the model_path class_labels_path input_blob output_cvg output_bbox ROS launch args?

https://github.com/dusty-nv/ros_deep_learning/tree/master#detectnet-node-1 https://github.com/dusty-nv/ros_deep_learning/blob/master/launch/detectnet.ros2.launch

[kleanbotmk2]

@kleanbotmk2 as I linked to above, there are ROS params exposed that allow you to set the dimensions of the input stream: https://github.com/dusty-nv/ros_deep_learning/tree/master#video_source-node

I tried to set the width and height (400x200) of the image using these params but it is not working. The image resolution remains at 640x320 (something like that.

Regarding your ONNX detection model, are you setting the model_path class_labels_path input_blob output_cvg output_bbox ROS launch args?

I tried to set the model_path and the class_labels_path but not the three others, what values should i enter for the input_blob, output_cvg and output_box ? I can't find the informations.

Thanks you really much for your awnser !

[dusty-nv]

I tried to set the width and height (400x200) of the image using these params but it is not working. The image resolution remains at 640x320

It has to be a resolution that your camera supports. Internally it will automatically be downscaled by jetson-inference to the resolution that the model needs, so you don't need to worry about that.

what values should i enter for the input_blob, output_cvg and output_box ?

The same ones that you used when running detectnet with your ONNX model like here: https://github.com/dusty-nv/jetson-inference/blob/master/docs/pytorch-ssd.md#processing-images-with-tensorrt

[kleanbotmk2]

thanks so much it's working !

Just is that normal that when i subscribe to the /detectnet/detections topic and i'm printing the id of is detected, there is nothing. I only have one label but i've noticed the same things and default model for example, the first label in the labels.txt file is nothing and the other labels are some signs like (< : H). Do you know how to fix that and get the real labels name in my case "juice_box" ? Thanks in advance

[dusty-nv]

Glad that you got it started running @kleanbotmk2 !

The detections topic a standard message type (vision_msgs::Detection2DArray) and this stores the class ID's as integers, not strings. To map the class ID to it's string name (as defined in your labels.txt), you need to subscribe to the detectnet node's vision_info topic (vision_msgs::VisionInfo)

That vision_info message in turn gives you the name of the ROS param containing the ID->classname map:

https://github.com/dusty-nv/ros_deep_learning/blob/9b5a668ba2cb24708d35eb13f734f343be223bd0/src/node_detectnet.cpp#L269

Read that ROS param and it will give you the list of class names (as strings). Sorry that is a convoluted process, but it's how it's done with the standard vision_msgs

[kleanbotmk2]

Hi @dusty-nv, I tried the solution you suggested yesterday. I listened to the /detectnet/vision_info topic and there's only one message published on this topic and that's when the detectnet node starts up. And I don't see any information about the classes in it.

And I would have preferred to have all the information on the vision_msgs/Detection2DArray topic so that the information could be synchronized, for example the id with the detection level and the bbox coordinates. Isn't there a way of obtaining the ids of the objects detected on this topic (for example, for the basic model supplied, the chair corresponded to the > sign, the only problem is that for my model, which is only able to detect one object, the latter corresponds to nothing, there's nothing written, and I don't know how to exploit this in a condition, for example to make a filter when I have several objects to detect.) Any ideas?

Thanks in advance!

Sach

[dusty-nv]

I listened to the /detectnet/vision_info topic and there's only one message published on this topic and that's when the detectnet node starts up. And I don't see any information about the classes in it.

That message contains the database_location, which is the name of a ROS param which contains the class name list

And I would have preferred to have all the information on the vision_msgs/Detection2DArray topic so that the information could be synchronized, for example the id with the detection level and the bbox coordinates. Isn't there a way of obtaining the ids of the objects detected on this topic

This does include the object class ID's (as integers). They are inside the Detection2D -> ObjectHypothesisWithPose

Although I agree it would be more straightforward just to have the class name strings inside the Detection2D message, vision_msgs are ROS standard and I can't change them. This is not specific to detectNet in any way.

[kleanbotmk2]

This does include the object class ID's (as integers). They are inside the Detection2D -> ObjectHypothesisWithPose

Although I agree it would be more straightforward just to have the class name strings inside the Detection2D message, vision_msgs are ROS standard and I can't change them. This is not specific to detectNet in any way.

But when i use the type() fonction after i've subscribed to the /detectnet/detections topic, it returns String, there is no integers it's just signs (> ! . H)

That message contains the database_location, which is the name of a ROS param which contains the class name list

What do you mean by ROS param, how do i exploit that ?

[dusty-nv]

I think maybe you are trying to access the wrong member in detections message?

ROS params: http://wiki.ros.org/rosparam

[kleanbotmk2]

okay, i think i'm starting to understand but when i tried to execute this line (to get the param): ros2 param get /detectnet/class_labels_15391228224745769807

I got this error: ros2 param get: error: the following arguments are required: parameter_name Do you know what's the parameter_name ? (and just so i understand, this should return me "juice_box" because it's what the model is what the detectnet node is detecting)

Thanks

[dusty-nv]

Try ros2 param get /detectnet class_labels_15391228224745769807

See here for ros2 param command-line guide: https://docs.ros.org/en/foxy/How-To-Guides/Using-ros2-param.html

It will give you list of all the class names. Then you index it by class ID

[kleanbotmk2]

alright so i used tis line : ros2 param get /detectnet/detectnet class_labels_15391228224745769807 and it has returned: String values are: ['BACKGROUND', 'juice_box']

But how do i use that in my python node ? (I'm trying to make my robot center itself on the object detected only if it's a "juice_box" detected, that's why I have to do some kind of filter so I need to retrieve the id)

[dusty-nv]

Here are some examples I found for getting params from another node using rclpy:

https://answers.ros.org/question/340600/how-to-get-ros2-parameter-hosted-by-another-node/ https://answers.ros.org/question/397679/how-to-setget-ros2-params-from-another-node-using-python/

[kleanbotmk2]

Thanks you very much but it the topic do not refresh, it just list the class from the labels.txt and that's it. I think i need to try to use the /detectnet/detections topic even if i don't know yet how to exploit the "empty" value in the id