Random scaling of mesh is required

[shruthi-autodesk]

As mentioned in this comment, I scaled my custom mesh such that it's in meters using trimesh.apply_scale(0.001). However, this scaling seems to make the mesh too small. It seems like, for each custom mesh I use, I have to apply an empirical scaling factor that converts my mesh to match the pointcloud from the scene. Here's an example:

Mesh scaled at 0.001 Picture of ground truth RGB image overlaid with the mesh in predicted pose

Picture of ground truth pointcloud overlaid with mesh pointcloud in predicted pose

Mesh scaled at 0.014 Picture of ground truth RGB image overlaid with the mesh in predicted pose

Picture of ground truth pointcloud overlaid with mesh pointcloud in predicted pose

Why does this happen? I'm attaching my data in the same format as the demo mustard example and my debug folders (for the 0.001 scale case and 0.014 scale case) as well. The code can be run by just providing the path to this data in run_demo.py, and adding a trimesh scaling factor using mesh.apply_scale(0.014) after line 29.

Is there a way to automatically determine this scaling factor? Please help me out with this, I've been trying to debug this for a long time.

[shruthi-autodesk]

@wenbowen123

[wenbowen123]

can you share your debug_dir folder? One possibility is your point cloud is in the wrong scale. You can measure its dimension in meshlab.

[shruthi-autodesk]

Here's my debug folder when I used a scaling factor of 0.014: https://drive.google.com/file/d/1sOCSi3nX2Yu02nJ1ojE8vAVsGA3bpBUY/view?usp=drive_link

Debug folder when I used a scaling factor of 0.001: https://drive.google.com/file/d/1Zn1wRyQv_2YXNdhNTd3cHDYfRQyOp2o0/view?usp=drive_link

Link to data used: https://drive.google.com/file/d/1kkBJwaU9gqivWKxD8es0lMNoKmZ6Ti3o/view?usp=drive_link

[wenbowen123]

@shruthi-autodesk your point cloud has the wrong scale. You can double check your depth saving format.

[shruthi-autodesk]

@wenbowen123 You were right, thanks for pointing this out. I was always assuming that something's wrong with my mesh. I was able to fix it by making sure my depth image is in mm.