search

dusty-nv / ros_deep_learning

Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT
866 stars 258 forks source link

Detector node net : custom topics overlay error #56

Open hokwangchoi opened 3 years ago

hokwangchoi commented 3 years ago

Hello dusty,

I am trying to run detector node with my custom image topic and encountered a problem.

Using, auto img_sub = ROS_CREATE_SUBSCRIBER(sensor_msgs::Image, "custom rostopic name", 5, img_callback); I subscribe to my custom rostopic(sensor_msgs::Image) image which is 720x540 image size.

The detection seems to be working but at the stage of publishing an overlayed image, it crashes with the following error which seems to be about the memory issue.

[ INFO] [1605629139.267617011]: detectnet node initialized, waiting for messages
[ INFO] [1605629143.360332135]: allocated CUDA memory for 720x540 image conversion
[ INFO] [1605629143.450974899]: detected 2 objects in 720x540 image
[ INFO] [1605629143.451798939]: object 0 class #1 (person)  confidence=0.732592
[ INFO] [1605629143.452022085]: object 0 bounding box (107.788712, 229.121368)  (141.429123, 300.089172)  w=33.640411  h=70.967804
[ INFO] [1605629143.452106825]: object 1 class #1 (person)  confidence=0.795658
[ INFO] [1605629143.452231887]: object 1 bounding box (160.328171, 231.065933)  (185.133057, 295.645020)  w=24.804886  h=64.579086
[ INFO] [1605629143.458095690]: allocated CUDA memory for 720x540 image conversion
free(): invalid pointer
[ INFO] [1605629143.619500453]: allocated CUDA memory for 720x540 image conversion

I tried to identify where the free() function is called but without success.

What could be the possible problem here?

dusty-nv commented 3 years ago

Hmm, I am not sure of what is causing the error, either.

Is is possible for you to try a different resolution or test video source to see if the problem is somehow related?

hokwangchoi commented 3 years ago

@dusty-nv I looked into it and debugged what's going wrong.

In the end, the problem was the overlay function which is called inside the imageCallback function.

I can send a pull request if you want :)

Thanks

dusty-nv commented 3 years ago

Interesting, thanks for debugging it. If you have the solution, feel free to submit a PR and I will take a look.

Unfortunately on my testing of ROS Melodic and ROS Eloquent, I did not experience this issue with detectNet overlay publisher.

jstumpin commented 2 years ago

@dusty-nv I looked into it and debugged what's going wrong.

In the end, the problem was the overlay function which is called inside the imageCallback function.

I can send a pull request if you want :)

Thanks

I can vouch for this on ROS Noetic. Getting "double free or corruption (!prev)" error calling the default publish_overlay from within img_callback. Pulling out the content of publish_overlay resolves the matter.

Thanks.

nikola-j commented 1 year ago

Same issue on ROS2 galactic, the PR fixes it.

Michsh commented 1 year ago

ROS Noetic,Jetson AGX, Jetpack 5.1, same error [ INFO] [1680256993.259650506]: node namespace => /detectnet [ INFO] [1680256993.259867286]: class labels => /detectnet/class_labels_1570648143953705978 [ INFO] [1680256993.285070984]: detectnet node initialized, waiting for messages [ INFO] [1680256993.578851178]: allocated CUDA memory for 640x480 image conversion [ INFO] [1680256993.626774664]: detected 1 objects in 640x480 image [ INFO] [1680256993.627043990]: object 0 class #8 (aaaa) confidence=0.866124 [ INFO] [1680256993.627174621]: object 0 bounding box (491.642334, 154.191620) (639.000000, 295.336273) w=147.357666 h=141.144653 [ INFO] [1680256993.633320713]: allocated CUDA memory for 640x480 image conversion double free or corruption (!prev)