Question about this code

[Shame-fight]

Is your yolov4_trt_ros based on the https://github.com/jkjung-avt/tensorrt_demos . I am looking for this version of ros packaging. Thank you for your work. Can your code run on jetson nano? Does it support yolov3? Is the ros version ros-melodic or ros2? Looking forward to your reply。 @indra4837

[indra4837]

Is your yolov4_trt_ros based on the https://github.com/jkjung-avt/tensorrt_demos . I am looking for this version of ros packaging. Thank you for your work.

Hi, yes this code is based on that repository. I adapted it to make it work with ROS and published it to my own message type defined in the msg directory

Can your code run on jetson nano?

It should work as long as the dependencies are met (i.e TensorRT 6+). I have listed the dependencies in the README.md.

Does it support yolov3?

I have used yolov3 on it and it does work. I have also added support for yolov3 in the recent commits. Do let me know if it works for you!

Is the ros version ros-melodic or ros2?

This is for ros-melodic.

Looking forward to your reply。 @indra4837

[Shame-fight]

I have meet a problem when run catkin_make: CMake Error at yolov4_trt_ros/CMakeLists.txt:43 (add_executable): add_executable cannot create target "video_source" because another target with the same name already exists. The existing target is an executable created in source directory "/home/knt/yolo_ros/src/ros_deep_learning". See documentation for policy CMP0002 for more details. @indra4837

[indra4837]

@Shame-fight
Hi, this package uses the same nodes as the ros_deep_learning package.

You can either,

1. Place a CATKIN_IGNORE file in the ros_deep_learning package to ignore building that package
2. Move that package outside of your catkin workspace

[Shame-fight]

Thank you for your prompt reply.I want to use usb camera,but I found that CSI camera is in package of ros_deep_learning,like
video_source.ros1.launch.Is there easy way to use USB camera directly? @indra4837

[indra4837]

Yes, the video_source.launch file in this package is exactly the same as the one in ros_deep_learning as i took it from there. You should be able to input the same arguments for the USB camera

[Shame-fight]

Thanks for your help, I have successfully run the code on jetson nano, but encountered two problems:

1. I use a usb camera, and the detection result has a delay of close to 3 seconds, which is unacceptable. I use v4l2///dev/video0 to read the camera video, width=640, height=480. I want to know how to reduce the delay. Do you know how to modify the camera reading frame rate (default 30), because the detection rate is only 4-5fps, so there is no need for such a high frame rate.
2. I used the command rostopic echo /detections to view the information of the released detection frame coordinates and other information, but an error was reported ERROR:Cannot load message class for [yolov4_trt_ros/Detectior2DArray].Are your messages built? I don't know the cause of this problem, can you help me? I was use yolov3.trt. Looking forward to hearing from you. @indra4837

[Shame-fight]

1.I have solved the first problem according to modify this part of the code of trt_yolo_v3.py: self.image_sub = rospy.Subscriber( self.video_topic, Image, self.img_callback) self.detection_pub = rospy.Publisher( "detections", Detector2DArray, queue_size=25) to self.image_sub = rospy.Subscriber( self.video_topic, Image, self.img_callback,queue_size=1,buff_size=1920*1080*3) self.detection_pub = rospy.Publisher( "detections", Detector2DArray, queue_size=1) The delay basically disappeared.But I still didn't find a place to modify the camera frame rate, the code defaulted to the highest frame rate. 2.I am working on fixing the second question. Can you run yolov3_trt.launch on jetson nano to confirm that there is no problem? I ran all the steps correctly, but this problem cannot be solved. Can you view the messages posted on the /detections topic? Thanks for your work .

[indra4837]

1.I have solved the first problem according to modify this part of the code of trt_yolo_v3.py: self.image_sub = rospy.Subscriber( self.video_topic, Image, self.img_callback) self.detection_pub = rospy.Publisher( "detections", Detector2DArray, queue_size=25) to self.image_sub = rospy.Subscriber( self.video_topic, Image, self.img_callback,queue_size=1,buff_size=1920*1080*3) self.detection_pub = rospy.Publisher( "detections", Detector2DArray, queue_size=1) The delay basically disappeared.But I still didn't find a place to modify the camera frame rate, the code defaulted to the highest frame rate.

Thats great, thanks for your work! Will modify the code to fix this. As for camera frame rate, let me take a look at it. It should be based on the jetson-inference/utils/camera/v4l2Camera.cpp (for USB camera).

2.I am working on fixing the second question. Can you run yolov3_trt.launch on jetson nano to confirm that there is no problem? I ran all the steps correctly, but this problem cannot be solved. Can you view the messages posted on the /detections topic? Thanks for your work .

I am able to get messages from the /detections topic when running yolov3_trt.launch.There is a small typo here in the error message ERROR:Cannot load message class for [yolov4_trt_ros/Detectior2DArray].Are your messages built?. It should be Detector2DArray instead. Could you try correcting it and see if it works?

[indra4837]

@Shame-fight

2.I am working on fixing the second question. Can you run yolov3_trt.launch on jetson nano to confirm that there is no problem? I ran all the steps correctly, but this problem cannot be solved. Can you view the messages posted on the /detections topic? Thanks for your work .

You can change the frame rates for CSI camera based on jetson-inference/utils/camera/gstCamera.cpp line 359 From :

if( mOptions.frameRate <= 0 )
    mOptions.frameRate = 30;

To:

if( mOptions.frameRate <= 0 )
    mOptions.frameRate = 10; // new frame rate

As for USB camera, I believe you need to take a look at the specific camera drivers as the jetson-inference/utils/camera/v4l2Camera.cpp file does not handle the frame rates. I'm not too familiar with USB cameras as I am currently using a CSI camera and can confirm the above method works for CSI camera frame rates. Let me know if you managed to do it, thanks!

[Shame-fight]

thank you for your reply.I solved the second problem by source /devel/setup.bash before executing rostopic echo /detections. I will deal with the issue of frame rate as soon as possible. Now I have servel questions:

1. Through rostopic echo /detections, I got the coordinates and other information, but the center x, y and size_x and size_y of the bbox are all negative numbers, I don't understand. Is the origin of the coordinate axis in the upper left corner of the image?
2. The secs and nsecs of the header are both 0. Is this correct?
3. Does your code support running on a device without a display? I use it on a robot without an external display. this is the information about rostopic /detections `--- header: seq: 72 stamp: secs: 0 nsecs: 0 frame_id: '' detections:

   • header: seq: 0 stamp: secs: 1601194201 nsecs: 674200057 frame_id: '' results: id: 0 score: 0.837864041328 bbox: center: x: -260.0 y: -158.0 theta: 0.0 size_x: -520.0 size_y: -315.0 `

[indra4837]

1. Through rostopic echo /detections, I got the coordinates and other information, but the center x, y and size_x and size_y of the bbox are all negative numbers, I don't understand. Is the origin of the coordinate axis in the upper left corner of the image?

Thank you for pointing this out, I have incorrectly indicated the boxes indexes. The boxes list is [xmin, ymin, xmax, ymax]. I will fix the code (self.publisher) appropriately as below.

        for i in range(len(boxes)):
            # boxes : xmin, ymin, xmax, ymax
            for _ in boxes:
                detection.header.stamp = rospy.Time.now()
        detection.header.frame_id = "camera" # change accordingly
                detection.results.id = clss[i]
                detection.results.score = confs[i]

                detection.bbox.center.x = (boxes[i][2] - boxes[i][0])/2
                detection.bbox.center.y = (boxes[i][3] - boxes[i][1])/2
                detection.bbox.center.theta = 0.0  # change if required

                detection.bbox.size_x = abs(boxes[i][0] - boxes[i][2])
                detection.bbox.size_y = abs(boxes[i][1] - boxes[i][3])

2. The secs and nsecs of the header are both 0. Is this correct?

It should be non-zero as follows,

3. Does your code support running on a device without a display? I use it on a robot without an external display.

Yes, it does. Do change the show_image to False to avoid viewing the detection results. You can subscribe to the /detections topic to get the detected classes and the /results/overlay to get the overlayed image with all the bounding boxes and classes.

[Shame-fight]

Hi, I'm bothering you again. I found that the center coordinate information of the bbox seems to be incorrect. I checked trt_yolo_v3.py, should I change lines 147-148 (approximate position)

detection.bbox.center.x = boxes[i][0]+(boxes[i][2]-boxes[i][0])/2 detection.bbox.center.y = boxes[i][1]+(boxes[i][3]-boxes[i][1])/2

At the same time, the image coordinate system is based on the upper left corner of the image as the origin, because I found that if the image appears below, center.y will be less than 240 (my image size is 640*480).

[indra4837]

Hi, I'm bothering you again. I found that the center coordinate information of the bbox seems to be incorrect. I checked trt_yolo_v3.py, should I change lines 147-148 (approximate position)

detection.bbox.center.x = boxes[i][0]+(boxes[i][2]-boxes[i][0])/2 detection.bbox.center.y = boxes[i][1]+(boxes[i][3]-boxes[i][1])/2

Yes, you are right. I missed out the minimum value in this line.

At the same time, the image coordinate system is based on the upper left corner of the image as the origin, because I found that if the image appears below, center.y will be less than 240 (my image size is 640*480).

Yupp, the origin is the top left corner of the image.

Thank you for informing me of the bugs you found. I have yet to test this code properly so there might be bugs here and there. Cheers!

[Shame-fight]

Hi.I found two problems:

1. The published bbox msg header:secs and nsecs are always 0
2. The same coordinate frame is often published twice

I didi not modify yolov3_trt.launch ,just modified video_source.launch to the following form `

<arg name="input_width" default="640"/>
<arg name="input_height" default="480"/>
<arg name="input_codec" default="unknown"/>
<arg name="input_loop" default="0"/>`

do I need modify yolov3_trt.launch? I do not know if this is the cause of the problem.

[indra4837]

1. The published bbox msg header:secs and nsecs are always 0

I forgot to input the timestamp for this. You can add it as below,

 for i in range(len(boxes)):
            # boxes : xmin, ymin, xmax, ymax
            detection2d.header.stamp = rospy.Time.now()
            for _ in boxes:
                detection.header.stamp = rospy.Time.now()
        detection.header.frame_id = "camera"
                detection.results.id = clss[i]
                detection.results.score = confs[i]

                detection.bbox.center.x = boxes[i][0] + (boxes[i][2] - boxes[i][0])/2
                detection.bbox.center.y = boxes[i][1] + (boxes[i][3] - boxes[i][1])/2
                detection.bbox.center.theta = 0.0  # change if required

                detection.bbox.size_x = abs(boxes[i][0] - boxes[i][2])
                detection.bbox.size_y = abs(boxes[i][1] - boxes[i][3])

            detection2d.detections.append(detection)

        self.detection_pub.publish(detection2d)

2. The same coordinate frame is often published twice

This is not much of an issue as I believe its due to the mismatch of FPS on your device with the input camera and the objects it is detecting are stationary/moving slowly. Thus, the image frames passed to the img_callback function are similar and it detects the same objects consecutively. The scores of the objects detected are different.

On my end, it works fine and the published messages are not duplicates.

[indra4837]

I'll close this issue now. Feel free to reopen it if you have any more questions, thanks!