search

j96w / DenseFusion

"DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion" code repository
https://sites.google.com/view/densefusion
MIT License
1.1k stars 300 forks source link

Testing the model #147

Closed ShrutheeshIR closed 4 years ago

ShrutheeshIR commented 4 years ago

This is great work! I was wondering if the pretrained model would work for real data, as well (an object that resembles an object in the datasets for training, but is not the same). To be more clear, I have a trained segmentation model that can segment out my shapes well. If I replace the segmentation module in your code, with mine, would the code still work to estimate the pose of such objects?

Suppose for example, if the code was trained on a cup, and if I run it on a different cup (such as a different color or small change in handle shape) would it still generalize and work well?

Apologies if my question is not framed appropriately!

Thanks!

huckl3b3rry87 commented 4 years ago

I don't think so. See this issue https://github.com/j96w/DenseFusion/issues/117

j96w commented 4 years ago

Hi @ShrutheeshIR, thanks for mention this good question. Your idea is category-level pose estimation while this work is designed for instance-level tracking (usually ppl in the industry have a specific set of target objects and require more accuracy on the pose estimation results). For category-level pose estimation, you can refer to our recent work 6-PACK, where the model is tested on unseen objects (within the training category).

j96w commented 4 years ago

Also, there are some methods to make DenseFusion become more robust to tiny appearance changes (similar but not the same as you mentioned). Basically, if you have the mesh model, you can randomize the texture of the object and generate synthetic dataset with it. The DF model trained by these data would have the potential to tackle the same object with unseen texture.

ShrutheeshIR commented 4 years ago

Hi @j96w , I will surely check out the 6-PACK work, seems interesting! Oh yes, I do have the mesh models, thanks for your suggestion, I will try it out and check the results. Thanks!