stefan-ainetter / SCANnotate

Repository for WACV23 paper "Automatically Annotating Indoor Images with CAD Models via RGB-D Scans"
GNU General Public License v3.0
13 stars 1 forks source link

Question about bounding boxes orientation in pytorch3d space #7

Open Alex-experiments opened 4 months ago

Alex-experiments commented 4 months ago

Hello,

I was wondering how the basis of the bounding boxes were transformed in your preprocessing step. Indeed, even though the example scene comes with a 'bbox_all.ply' mesh showing the bounding boxes on the preprocessed mesh, when I try to manually place boxes in Blender with the given center, scale and orientation stored in the scene pickle file, the boxes are off...

For instance, I tried to place two tables which have a non zero angle along the Y axis: bbox_base

After trying to understand what was off, it seems like simply inverting the Y axis' rotation allows for a near perfect alignment of the placed boxes and the original bounding boxes: bbox_modified It is visually near perfect, although when looking closely the corners of the box don't line up perfectly with yours. We can see that in the following screenshot where the top left corner doesn't really align and the edge at the top isn't perfectly parallel to yours. (Sorry for the random orange circle, it is a blender tool) corner_weird

It's weird to me as I was thinking that it was some some of change in the coordinates systems that would induce that, but I don't think such a change would induce a "mirroring effect" of the Y axis rotation. Am I doing something wrong here or is there a step I didn't understand?

For reproducability, I imported "mesh_py3d_textured.ply", "bbox_all.ply" in Blender, and with a small script read the center, basis and scale of these two tables. I then proceeded to place cubes in Blender at the given position with the given scale (halved as Blender spawns cubes that are of size 2) and the given rotation (converted from a 3x3 rotation matrix to euler angles in degrees).