I have a question to get the yolo_output TF value by realsense D435.

[hyunoklee]

hello. Over the past few months, I've been using your code very well on a Gazebo simulation base. Thank you. And Recently I bought RealSense D435, But There is some problem.
I've been debugging for a few weeks and I can't solve the probolem. I would really appreciate it if you answered my question.

My Goal is to get the yolo_output TF value by realsense D435.

I run below command to get yolo_output TF value by RealSense D435.

roslaunch turtlebot3_slam_3d turtlebot3_d435_bringup.launch roslaunch turtlebot3_slam_3d darknet.launch generate_point_cloud:=true use_zed:=false So, I can not get yolo_output TF value. TF tree is below.

I thought that yolo_output should come after camera_color_optical_frame , but I do not see it. base_link->camera_link-> camera_aligned_depth_to_color_frame-> camera_color_optical_frame -> nothing https://github.com/hyunoklee/test/blob/master/frames_real.png?raw=true

Is there anything I need to addition modify to get the yolo_output TF value by D435 ?

Should I add the following command to turtlebot3_d435_bringup.launch ? <node pkg="tf" type="static_transform_publisher" name="tf_transform" args="0 0 0 0 0 0 /camera_depth_frame /camera_color_frame 100"/>

AND At /.rosinstall file, Use the without_tf_publish version like below.

/////// /.rosinstall //////////////////////////////////// uri: https://github.com/Affonso-Gui/realsense.git version: without_tf_publish

Unlike Zed Mini, I would really appreciate it if you let me know why you should use without_tf_publish version at D435.

Actually I tried already all of the things mentioned above. But I can not get yolo_output TF value by realsense D435. please help.

[Affonso-Gui]

Hello,

Thanks for your question. Since we are attending to an conference this week I cannot test this problem thoroughly right now, but will put my hands on it as soon as we come back.

What I can say for now is:

• Do not use generate_point_cloud:=true when connecting directly to the camera. This argument is only for point cloud reconstruction from bag files
• If you are not using rtabmap, try removing these lines from darknet.launch: (we have no map frame)https://github.com/ROBOTIS-JAPAN-GIT/turtlebot3_slam_3d/blob/fab7dd6b72de7421a9f618d67c4acafe82aba5dc/launch/darknet.launch#L52-L55
• The reason why we use the Affonso-Gui/realsense without_tf_publish branch is because the tf from D435 was interfering with the one published by the turtlebot. For ZED, using publish_tf:=false was enough. I will, however, check if some updates in the origin branch make this obsolete.

Debugging by steps:

• There seems to be no problems with the tf frames you are showing
• Can you see the camera color & depth images normally?
• When launching darknet.launch, do you see a popup with live camera viewing and detected objects?
• Can you subscribe darknet_ros/label_image?
• Can you subscribe cluster_decomposer/centroid_pose_array?

[hyunoklee]

Thank you very much for your reply. I learned a lot because of your answer. I understand Why you use without_tf_publish version at D435.

Now I can get yolo_output TF by D435 . I did not modify anything in your code. Just I had done the following command as stated in the Readme. roslaunch turtlebot3_slam_3d turtlebot3_slam_3d.launch use_zed:=false And I can get yolo_output TF by D435. I think your code is perfect .

I do not know exactly why I could not get yolo_output TF by D435 during a few week. Maybe , As your response, I think the reason is generate_point_cloud:=true. Now I use generate_point_cloud:=false, Operating is OK .

Thank you very much for the friendly explanation. d(･∀･○)

[Affonso-Gui]

Good to hear it is working again. Maybe some recent updates solved your problem.

For future reference, I am posting below a link to the jsk_pcl_ros documentation (the part responsible for publishing the tf from the label image):

https://jsk-recognition.readthedocs.io/en/latest/jsk_pcl_ros/nodes/cluster_point_indices_decomposer.html