Question of transformation of coordinates

[wqdong8]

Hello, this is a very meaningful work! But I have some problems with the transformation of the coordinate system in "convert_to_nerf_convention.py". Which is the up vector in the world coordinate system? What is the meaning of the zxy_to_xyz matrix name in the convert_pose_from_nerf_convention function? Thanks.

[lukasHoel]

Hey,

the purpose of convert_to_nerf_convetion.py is to transform world-to-cam matrices in the Pytorch3D convention to cam-to-world matrices in the standard NeRF convention and vice-versa.

• In our codebase we use the Pytorch3D convention to define cam-to-world matrices: https://pytorch3d.org/docs/cameras

• We follow conventions laid out by the original NeRF work and adopted by Instant-NGP and NeRFStudio codebases to transform these cameras to a quasi standard convention for NeRF: https://docs.nerf.studio/en/latest/quickstart/data_conventions.html

• convert_pose_to_nerf_convention and convert_pose_from_nerf_convention are the two functions that convert from one direction to the other (i.e., they are the exact inverse).

• The xyz_to_zxy matrix is needed, because the NeRF convention defines the XY plane parallel to the ground plane. Consequently, zxy_to_xyz is reverting this again when converting back to Pytorch3D.

Maybe if you give a more concrete example what does not work for you, I could help you figure it out. But these are the general comments about the methods.