monniert / differentiable-blocksworld

[NeurIPS 2023] Code for "Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives"
https://www.tmonnier.com/DBW
MIT License
197 stars 10 forks source link

The position of reconstructed mesh in quali_eval #10

Closed lw-ethan closed 5 months ago

lw-ethan commented 7 months ago

Hi, thanks for your nice work and concise repo!

I'm working on the project that needs to leverage on the reconstructed mesh generated from dbw, and I found the position and scale of mesh have certain gap compared to points computed from colmap. I'm wondering if the mesh is not placed on the real position of world coordinates, and instead always placed at the origin?

monniert commented 5 months ago

Hi @lw-ethan, the scene is indeed always placed at the origin. It is transformed to the colmap space using a predefined rotation, translation and scale matrix

lw-ethan commented 5 months ago

@monniert, thanks for your reply, I found that it was my mistake: I kept stuck at the configuration of S_world, R_world, T_world, which should be obtained easily by delta transformation in Blender I came to realize later the problem arise from the settings of R_world, the mesh applies world coordinate this way verts = (verts * self.S_world) @ self.R_world + self.T_world[:, None], the vertices are at the left side of R_world However I used the convention that R = RzRyRx * verts, which caused the transformation wasn't as my expect Changing the conversion R_world = (elev_to_rotation_matrix(elev) @ azim_to_rotation_matrix(azim) @ roll_to_rotation_matrix(roll))[None] to R_world = (roll_to_rotation_matrix(roll) @ azim_to_rotation_matrix(azim) @ elev_to_rotation_matrix(elev)).T[None] met my assumption and solved my problem