search

zaiweizhang / H3DNet

MIT License
211 stars 25 forks source link

[Help wanted] How to get the tilt angle for a new RGBD image? #6

Closed xiaodongww closed 3 years ago

xiaodongww commented 4 years ago

Hi, thanks for your great job. Here is a question about the training process of SUN-RGBD. I noticed that the axis is aligned to the gravity direction. The tilt angle is provided in your processed dataset. This means if I want to test the model with a new RGBD image taken from Kinect v2 sensor, I will also have to calculate the tilt angle. Do you have any idea how to do this? Are there any tools out of the box?

Any suggestions will be helpful. Thanks very much!

zaiweizhang commented 4 years ago

Hi,

This is a great question. I am actually very curious about it as well. However, I do not have the answer. SUN-RGBD image frames come with a calibration R tilt matrix. I have already emailed shuran, SUN-RGBD's author, about this issue. I will let you know once I get some updates.

Thanks, Zaiwei

xiaodongww commented 4 years ago

Thanks for your quick reply. I just checked the original SUN-RGBD paper. As you said, it did mention the alignment process as follows:

For 3D annotation, the point clouds are first rotated to align with the gravity direction using an automatic algorithm. We estimate the normal direction for each 3D point with the 25 closest 3D points. Then we accumulate a histogram on a 3D half-sphere and pick the maximal count from it to obtain the first axis. For the second axis, we pick the maximal count from the directions orthogonal to the first axis. In this way, we obtain the rotation matrix to rotate the point cloud to align with the gravity direction. We manually adjust the rotation when the algorithm fails.

It also mentioned that the algorithm may fail and need to manually fix the failure cases. I was wondering how often this algorithm fails.:joy:

zaiweizhang commented 3 years ago

I checked their provided SUN RGBD toolkit and I did not find the code for their automatic algorithm. I will email the author for this algorithm. I am also curious about how often this algorithm fails. I will let you know if I have some updates.

Thanks for your interest in our work btw!

Zaiwei

zaiweizhang commented 3 years ago

I still did not get any response from the author.

xiaodongww commented 3 years ago

@zaiweizhang Thanks for your help anyway. I will try it later and will let you know if I found something useful. For now, I am closing this issue.