search

huangjh-pub / multibody-sync

[CVPR'21 Oral] MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
59 stars 5 forks source link

Question: Orientation/Normalization of the input data #8

Closed SamMaoYS closed 2 years ago

SamMaoYS commented 2 years ago

Hi, thank you for the great work, and for sharing the code. I have a question about the orientation of the input data.

I downloaded the preprocessed train-val mbs-shapepart data for the articulated objects you shared in the repo. When I was visualizing the data, I found the orientation of the objects are neither y axis up nor -y axis up. Following are the visualizations I got from the data. (red: x axis; green: y axis; blue: z axis)

As you can see, there seems to be a 45-degree angle between the objects' up axis and the global y axis. I think you also mentioned, "normalize point cloud (preferably with -Y axis up) beforehand to get the best result.". Thus, I want to ask, if the observations are valid and if there is a specific reason to orient the objects to these poses.

In addition, if I want to test the pre-trained model with my own real-world objects, do you have suggestions on how should I orient/normalize my objects. And if there is any normalization and transformation pre-process on the DynLab Dataset for the evaluations.

Looking forward to hearing from you soon, and thank you for your time and help.

heiwang1997 commented 2 years ago

Thanks for your questions and here are my answers.

Q: If the observations are valid and if there is a specific reason to orient the objects to these poses. A: Yes. During training we orient our objects in such a way and it would be good if your test set follows a similar data distribution. This will minimize the domain gap and hopefully leads to a better generalization.

Q: How should I orient/normalize my objects. We are sorry for the confusion, as we generate a virtual camera during the data generation process. The camera looks down, thus resulting in the 45-degree tilt seen in your visualization. During the test with other out-of-category articulated objects, we find that such a small tilting will not greatly affect our results. Hence my suggestion is still -Y axis up, and try rotating your point clouds only if the results look bad.

Q: If there is any normalization and transformation pre-process on the DynLab Dataset During the evaluation, we follow the metric as in the provided dataset (not meter).

Hope it helps!