data['camera_to_aligned_camera'],planes_gt coordinate transformation and plane distance loss

[wangshuailpp]

hey@ymingxie, Thanks for your work! I have some questions to ask you for,Looking forward to your answer!

• about data['camera_to_aligned_camera'], I find neuralrecon do not have this transformation. however planarrecon save it and use it for plane_gt, I can not understand this code: https://github.com/neu-vi/PlanarRecon/blob/00bd50ccef849cc5cf0746ec6e6432e47a000750/datasets/transforms.py#L152-L165
• about plane distance loss, I find plane_gt transform by data['camera_to_aligned_camera'], but r_coords transform by data['world_to_aligned_camera'], Is there a problem with the calculated point and plane loss? https://github.com/neu-vi/PlanarRecon/blob/00bd50ccef849cc5cf0746ec6e6432e47a000750/models/planarrecon_network.py#L248-L251 Thanks!

[ymingxie]

Hi, this matrix is to convert the plane parameters from camera coordinate to a gravity-aligned coordinate :) The matrix is generated here: https://github.com/neu-vi/PlanarRecon/blob/main/datasets/transforms.py#L65. Because in ScanNet world coordinate the inverse z axis is gravity direction, we can generate the gravity-aligned coordinate by using the transformation from camera to world coordinate. NeuralRecon also has this transformation (https://github.com/zju3dv/NeuralRecon/blob/master/datasets/transforms.py#L79)

The plane gt and r_coords are both in the aligned_camera (gravity-aligned) coordinate. It should be OK.

Feel free to reach out to me if you have more questions :)

[wangshuailpp]

Thanks! I got it!