Open dnlwbr opened 2 years ago
@dnlwbr did you find anything related to your above questions?
Unfortunately, it's been some time since I've dealt with that and the whole thing is also somewhat ambiguous. If I remember correctly, however, these were my findings:
poses_bounds.npy
in drb format, because this line still does the transformation to OpenGL format. If the code was already in OpenGL format from the beginning, as recommended by the Readme, then the mentioned line would break everything. At least my results seem to have confirmed my assumption at that time.However, my statements are to be enjoyed with caution, since I am everything else than an expert.
Thanks @dnlwbr! This is helpful. I'll update here if I figure out anything more.
@dnlwbr I figured out what is happening w.r.t. your point 2 (Convention for translation isn't changed). You were right. Convention of both rotation and translation are changed. For the benefit of others who may also stumble upon this issue, I'll note down what I've understood so far.
R
from one convention to other, we find the corresponding permutation matrix P
and find the new rotation matrix as P' R P
. Similarly we find the new translation as P t
. But when we compute relative poses, right multiplication by P
is unnecessary (since it cancels out). So, we can multiply only on the left by P'
. Here, they first take inverse of the camera poses then multiply rotation matrix with the permutation matrix on the right with P
. Then they compute relative pose and take the inverse of this relative pose. When we invert the pose, rotation matrix is transposed so that R' P
becomes P' R
which is what we want. And the translation becomes -(R' P)' (-R' t)
which is equal to P' t
. Thus both rotation and translation are converted to the final convention.
Hello and thanks for sharing this nice work! I am still a bit confused by coordinate systems and the Readme. While the Readme says that
it seems that the LLFF code which is used by
image2poses.py
only transforms the rotation matrix from [right, down, forward] (COLMAP) to [down, right, backward] and NeRF later converts to [right, up, backward] (OpenGL).poses_bounds.npy
with own poses as suggestes in the Readme (OpenGL format), then this line would still apply the conversion to OpenGL format. That doesn't seem right to me, or am I misunderstanding something?poses_bounds.npy
should still be in COLMAP format. Is that correct?Thanks in advance!