Has anyone compared to SAM6D?

[GavinYang5]

I've tried comparing some of the more heavily occluded scenes, and SAM6D seems to work better than FoundationPose Unfortunately, my computer broke down and I didn't save the result image... Has anyone made some comparisons? By the way, I only have a textureless mesh of objects, will this affect the effect?

[GavinYang5]

@wenbowen123

[wenbowen123]

textureless mesh

if the object has texture in reality, but your mesh does not have texture, that would certainly affect the results, cause FoundationPose will also leverage RGB cues.

[GavinYang5]

Oh, I understand, but SAM6D works well in the same situation. I wonder if I haven't set it up correctly regarding FoundationPose? Can you explain the proportion of texture information's impact on FoundationPose? Later (after I fix my computer), I will display some comparison images of the effects on my custom dataset.

[wenbowen123]

See this example how FoundationPose distinguishes the rotation of the tomato can by using the RGB cues. If the mesh's color deviate a lot from the scene (e.g. untextured mesh), it will cause a big performance drop.

What objects are you using and what prevents you of getting the texture?

[GavinYang5]

Thank you for your reply. Yes, I have looked at the example and the effect is great. It seems that RGB information is still quite important in FoundationPose. Regarding my Mesh, I used a 3D camera to capture my object point cloud and GT Pose from multiple perspectives, then concatenated them and obtained them using o3d.create_from_point_cloud_poisson. Regarding normal, I used o3d.compute_triangle_mormals and o3d.compute_vertex_mormals.

[wenbowen123]

the way you get the mesh is really suboptimal. You can use the model-free setup provided in this repo, or you can explicitly build a textured mesh using BundleSDF.

[GavinYang5]

Yes, I'll try BundleSDF later, if there are other questions, I'll consult you again, thanks for the answer