Custom Data, vis_pem.png not matching.

[kevinDrawn]

Hello, I'm struggling to adapt the sam-6d on my custom data.

ISM is work perfectly. But PEM is abnormal.

why this PEM precision is worst?

My 3D Model is normal.

Can you suggest some idea? @JiehongLin

[foollh]

It seems to me that you should check the units of CAD and depth. The input CAD and depth in this demo are both in millimeters.

[kevinDrawn]

Thank you for reply. I checked my CAD and Depth images. There were two problem.

1. vertex unit of CAD was set to 'm'. So I changed to 'mm'.
2. My Depth Camera(D455) scale is 0.001. But, it makes worst.

whole_depth = load_im(depth_path).astype(np.float32) * cam_info['depth_scale'] / 1000.0 When I set my depth_scale to 0.001, whole_depth value was nearly 10^-6. It is very different with example depth data which has 10^-3. So I changed my depth_scale 0.001 to 1.

But PEM precision didn't improve.

I have attached my input model.

Thank you a lot!

input.tar.gz

[foollh]

I have evaluated the data you provided, and the pose estimate is exactly accurate, as shown in the figure below. The yellow points represent the point cloud of CAD transformed by the predicted pose, and align with the red points cropped from the observed point cloud.

The 3D bounding box seems to be wrong because the original CAD is not semantically aligned, as shown in the figure. We have added the visualization code for the model point projection. You can try it again.

[kevinDrawn]

Wow, amazing. Thank you so much! Does this mean that even if there's an issue with the visualization box, the Pose Estimation Values (R, T) are still correct? And the red dots represent the expected 3D Model Point?

I checked the 'scale' in Pose_Estimation_Model/run_inference_custom.py, and it's set to scale = np.max(np.abs(model_points), axis=0)2 1000. The output scale was [295.84, 170.18, 408.63]. However, the dimensions of my 3D model are [210, 240, 65] mm. I think the system might have sampled points in the wrong place. Could this have caused a semantic issue? Thank you so much!

ps. How can you evaluate my cad data? Is is provided in this repo?

[foollh]

Yes. The semantic issue means the 3d bounding box calculated in the current CAD coordinate system does not closely fit the mesh. You can change the CAD coordinate system before input to solve this problem.

The data I evaluated are from the link you gave in this comment.

Thank you for reply. I checked my CAD and Depth images. There were two problem.

1. vertex unit of CAD was set to 'm'. So I changed to 'mm'.
2. My Depth Camera(D455) scale is 0.001. But, it makes worst.

whole_depth = load_im(depth_path).astype(np.float32) * cam_info['depth_scale'] / 1000.0 When I set my depth_scale to 0.001, whole_depth value was nearly 10^-6. It is very different with example depth data which has 10^-3. So I changed my depth_scale 0.001 to 1.

But PEM precision didn't improve.

I have attached my input model.

Thank you a lot!

input.tar.gz

[kevinDrawn]

Thank you so much! I solved my problem!