search

blodow / realtime_urdf_filter

ROS package that can filter geometry defined in URDF models from Kinect depth images. Can also preprocess data for the OpenNI tracker, to remove backgrounds, robots etc.
Other
89 stars 46 forks source link

filtered Sawyer missing links #44

Open yufeiwang63 opened 1 year ago

yufeiwang63 commented 1 year ago

Hi, Thanks for this great packge!

I am using this for filtering the sawyer robot. However, I found that the filtered results are missing some of the sawyer links -- it only filters the base and the pedestal: image

However, Rvis can correctly load and render the full urdf: image

Here is my launch file, modified to be used with RealSense:

<launch>
  <arg name="nodelet" default="false"/>

  <!-- Launch nodelet -->
  <node if="$(arg nodelet)" pkg="nodelet" type="nodelet" name="realtime_urtf_filter_nodelet" args="load realtime_urdf_filter/RealtimeURDFFilterNodelet camera_nodelet_manager" output="screen">
    <remap from="~input_depth" to="/camera/aligned_depth_to_color/image_raw"/>
    <remap from="~output_depth" to="/camera/aligned_depth_to_color_filtered/image_raw" />
    <remap from="~output_mask" to="/urdf_filtered_mask" />

    <rosparam command="load" file="$(find realtime_urdf_filter)/launch/filter_parameters.yaml"/>
  </node>

  <node unless="$(arg nodelet)"
    pkg="realtime_urdf_filter" type="realtime_urdf_filter" name="$(anon realtime_urdf_filter)" output="screen">
    <remap from="~input_depth" to="/camera/aligned_depth_to_color/image_raw"/>
    <remap from="~output_depth" to="/camera/aligned_depth_to_color_filtered/image_raw" />
    <remap from="~output_mask" to="/urdf_filtered_mask" />

    <rosparam command="load" file="$(find realtime_urdf_filter)/launch/filter_parameters.yaml"/>
  </node>

  <!-- Load an example URDF -->  
  <!-- <param name="robot_description_2" command="$(find xacro)/xacro $(find sawyer_description)/urdf/sawyer_gpt.urdf.xacro electric_gripper:=false"/> -->
  <!-- <param name="robot_description_2" textfile="sawyer_description/urdf/sawyer.urdf"/> -->
  <param name="robot_description_2" textfile="sawyer.urdf"/>

  <node name="robot_state_publisher" pkg="robot_state_publisher" type="robot_state_publisher">
    <param name="tf_prefix" value="filter_sawyer"/>
  </node>
</launch>

And here is my filter_parameters.yaml:

fixed_frame: /base
camera_frame: /camera_depth_optical_frame
camera_offset:
  translation: [0.0, 0.0, 0.0]
  rotation:    [0.0, 0.0, 0.0, 1.0]
# There is one entry for each URDF that should be filtered
models:
- model: "robot_description_2"
  tf_prefix: "/filter_sawyer"
  geometry_type: "visual" # "visual" or "collision"
  scale: 1.0
  ignore: []
# how far in front of the robot model is still deleted? (e.g. 0.05 = 5cm)
depth_distance_threshold: 0.05
show_gui: true
filter_replace_value: 0

Any suggestions on why this is happening would be appreciated!

yufeiwang63 commented 1 year ago

here is the sawyer urdf for your reference (converted to .txt so it can be uploaded to github sawyer_urdf.txt )

blodow commented 1 year ago

Hey, glad you like it!

Similar questions arise from time to time, and some if not most of them can be answered with “the code loading the models/meshes could be improved”. One common issue is the presence of faces in the model that have more than 3 vertices, e.g. quads and higher order faces. There can probably be other issues preventing correct loading, like collada features that the code is not dealing with correctly. I would try to see if i can load the missing meshes in e.g. MeshLab, triangulate everything and see if that fixes it.

However, ideally some one would at some point step in and attempt to improve the loading code to make life easier for everyone, but it seems fixing it by converting the file is preferable to most. I also don’t have the resources to take care of it, so I’m grateful for every PR that fixes these sorts of problems.

On Sat 22. Apr 2023 at 20:58 yufeiwang63 @.***> wrote:

Hi, Thanks for this great packge!

I am using this for filtering the sawyer robot. However, I found that the filtered resutls are missing some of the sawyer links -- it only rendres the base and the pedestal: [image: image] https://user-images.githubusercontent.com/37777198/233801765-1e3b1d9d-3acb-4ec0-a80e-6137acd6a743.png

However, Rvis can correctly load and render the full urdf: [image: image] https://user-images.githubusercontent.com/37777198/233801807-2866bc5e-414e-4473-8871-64c78867d80c.png

Here is my launch file, modified to be used with RealSense:

And here is my filter_parameters.yaml:

fixed_frame: /base camera_frame: /camera_depth_optical_frame camera_offset: translation: [0.0, 0.0, 0.0] rotation: [0.0, 0.0, 0.0, 1.0]

There is one entry for each URDF that should be filtered

models:

  • model: "robot_description_2" tf_prefix: "/filter_sawyer" geometry_type: "visual" # "visual" or "collision" scale: 1.0 ignore: []

    how far in front of the robot model is still deleted? (e.g. 0.05 = 5cm)

    depth_distance_threshold: 0.05 show_gui: true filter_replace_value: 0

— Reply to this email directly, view it on GitHub https://github.com/blodow/realtime_urdf_filter/issues/44, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADW24QQP4WJB56LJJLTVKLXCQS55ANCNFSM6AAAAAAXIADRAA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

yufeiwang63 commented 1 year ago

Thanks for your quick reply. However, if I change the loading of the urdf from visual to collision, and I manually commented all the visual property, i.e., all the mesh loading part of the urdf file, it still does not load correctly, so I guess it should not be a mesh loading issue?

image

Do you have any more suggestion on why this might still happen? Thanks!

blodow commented 1 year ago

Both visual or collision elements can be defined as meshes. I saw in your text file that the meshes are in fact collada (.dae) files, which is a very complex format. We are using assimp to load these files, but that code is not covering all it should cover. Check out RenderableMesh::initMesh in renderable.cpp, there is a fixme regarding vertex transforms in collada files, and an assert about 3 vertices. I would expect that if you step through this with a device or add debug prints, it would complain about your problematic files. As I said, the quick fix is to simplify the mesh files so the code loads it correctly, the better fix would be to improve the mesh loader so it can load more files successfully.

On Sat 22. Apr 2023 at 21:17 yufeiwang63 @.***> wrote:

Thanks for your quick reply. However, if I change the loading of the urdf from visual to collision, and I manually commented all the visual property, i.e., all the mesh loading part of the urdf file, it still does not load correctly, so I guess it should not be a mesh loading issue?

[image: image] https://user-images.githubusercontent.com/37777198/233802532-eb3957ee-cb24-48e9-87ec-e53765dc5245.png

Do you have any more suggestion on why this might still happen? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/blodow/realtime_urdf_filter/issues/44#issuecomment-1518737622, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADW24XAX2YOCMFXIC3KZKDXCQVEDANCNFSM6AAAAAAXIADRAA . You are receiving this because you commented.Message ID: @.***>