Use in ROS melodic+UR5+KinectV2 has a problem？The correct image always flashes for a short time, and then the wrong image appears.

[nemoxiaoli]

Hi all good man,i am sorry for disturbance.@JimmyDaSilva @blodow Thank you for share us this wonderful package .I am a new novice of Ros.I want to filter the image of robot from the depth image to track the moving obstacle.Luckly this package seems to provide such function. However,when I run this package, the output image is wrong.The correct image always flashes for a short time, and then the wrong image appears.I have worked on this for many days,and i can not work out this question .So,i ask you for a help.Thank you again. I have tried to change the filter_kinectV2.launch and the filter_parameters.yaml for my system . The changed filter_kinectV2.launch file: The changed filter_parameters.yaml file:

I start these files for running this package: 1.roslaunch ur_modern_driver ur5_bringup.launch limited:=true robot_ip:=192.168.250.1 use_lowbandwidth_trajectory_follower:=true 2.roslaunch ur5_moveit_config ur5_moveit_planning_execution.launch limited:=true 3.roslaunch ur5_moveit_config moveit_rviz.launch config:=true 4.roslaunch kinect2_bridge kinect2_bridge.launch publish_tf:=true 5.roslaunch easy_handeye start.launch 6.roslaunch realtime_urdf_filter filter_kinectV2.launch

The video of the output image as follows: https://user-images.githubusercontent.com/97078679/226268932-744cbd10-4d31-4ab1-a7ac-e7086cc1d55f.mp4 the right image(just maintain a short time): the wrong image(appear for the most time):

What else should I do to run this package? Any suggestion would be grateful

[blodow]

Hi @nemoxiaoli, I am glad you like the URDF filter :D

In short, there are two main things that must be correct for URDF filter to be able to work:

1. The coordinate transforms in your system need to work. This means that there must be a valid chain of tf frames between the camera frame and the robot frames, otherwise we cannot know where the robot geometry needs to be rendered from the virtual camera of the URDF filter. It is important that you provide the correct frame names in the filter_parameters.yaml file or via launchfile / rosparam.
2. The robot model needs to be loaded by the URDF filter, and some model files don't work well with assimp or our assimp wrapping code. E.g. some binary STL files needed to be converted to some other format in the past.

It seems both things are there, because the initial render works. Why or when does it turn bad? Is it with the first transform update from tf? is fixed_frame and camera_frame set correctly? Can you visualize robot AND camera frame in RViz?

[blodow]

BTW @nemoxiaoli , the video you posted is corrupt and does not play.

[nemoxiaoli]

Thank you for the kindful response.

The things you mentioned is helpful，the details as follows:

1、I have set the chain of tf frames between the camera frame(kinect2_link) and the robot frame(base_link), the roslaunch easy_handeye start.launch include this:

2、The robot model is UR5, i think the URDF filter is suitable.

3、Sometimes, after the program starts, the correct image flashes, and then a wrong image appears. The same as you say "it is turn bad with the first transform update from tf". Another situation，the correct image and the wrong image are displayed alternately. I put the video below.

4、 I can visualize robot AND camera frame in RViz , please look at the picture and video below.

Here are the videos and picture,maybe you can open them in windowns, i find the video posted are corrupt in Ubuntu , but can play in windowns.

https://user-images.githubusercontent.com/97078679/227154082-8f3df01d-9f9b-44dc-b9df-38b3e45d45f6.mp4

https://user-images.githubusercontent.com/97078679/227154274-6bd0da93-18c7-4c12-acb0-9a1060343e29.mp4

[blodow]

Could it be that you have multiple nodes that publish tf transforms? They would conflict and send contradicting information to the URDF filter. It could maybe explain why it alternate between good and bad..

If none of these issues show up in rviz, then it makes a bug in our filter more likely.

[blodow]

Are there relevant lines in the log output of the filter? Does roswtf show something interesting? Can you run the minimal amount of components (tf publisher, urdf filter, kinect) and see if it still happens? rosnode list, rostopic list?

[nemoxiaoli]

Thank you for you aid . This time i just run 4 components(ur_driver , kinect , the chain of tf frames between kinect2_link and base_link, urdf filter) the roswtf,rosnode list,rostopic list as follows: