search

tomato1mule / edf

[ICLR 2023] Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning
MIT License
32 stars 1 forks source link

A question about the data #1

Closed wzm2256 closed 1 year ago

wzm2256 commented 1 year ago

Hi, thanks for your work! I'd like to ask a question about the demo data you use.

I visualized the data in folder demo/test_demo/data. I plot the scene point cloud, the grasp point cloud and the grasped point cloud transformed by the target pose, but it seems that the transformed grasped mug is not hung to the stand, which I think is absurd because I think the demo should be accurate. Do I get it right? or do I miss anything? Thanks for your time!

Here is my visualization. Blue: scene Green: transformed grasped orange: grasped

image

tomato1mule commented 1 year ago

Hi, wzm2256, I appreciate your interest in our work. Yes, the grasped mug should be displayed as being hung on a hanger if correctly visualized. It looks like something has gone wrong with the visualization.

First of all, it seems like the visualized demo is not the one in demo/test_demo, but the one from demo/test_unseen_demo. It is possible that I have put arbitrary target pose, because demo/test_unseen_demo is basically a data for validation, not for training. (It has been a while so I am not sure I remember correctly...). Please check if you are correctly visualizing data from demo/test_demo, not the demo/test_unseen_demo.

Another possibility is that you are using (qx,qy,qz,qw) convention for the quaternions. Our implementation follows the (qw,qx,qy,qz) convention for quaternions as PyTorch3D does. Please check again that your visualization code follows (qw,qx,qy,qz) formalism. For the whole SE(3) pose, we use (qw,qx,qy,qz,x,y,z) convention, which can be easily confused with (x,y,z,qw,qx,qy,qz).

We also provide a visualization method for 'TargetPoseDemo' class as well as 'PointCloud' class. You can visualize them with demo.show() or pcd.show() method.

Please let me know if the problem persists. Thank you!

wzm2256 commented 1 year ago

Oh, I see. Thanks! It works when I use the correct order of se3 pose.