Pointcept / OpenIns3D

[ECCV'24] OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation
MIT License
136 stars 9 forks source link

About the local Enforced Look Up Module #10

Closed LilRedWu closed 2 months ago

LilRedWu commented 4 months ago

Thank you for the excellent and insightful work! The paper introduces the Local Enforced Look Up Module, which enhances the proposed mask by calculating the occlusion rate to refine the mask further before passing it to a 2D detector for reclassification. However, I was unable to locate this module in the code, specifically within the pcd2img_point_occlusion_aware function. It's possible I might have missed it. Could you please help me identify which lines of code correspond to the LEL Module? image

ZheningHuang commented 4 months ago

Hi,

Thank you for your interest in our work and for testing out the code.

Regarding the LEL module, it yields an improvement in final performance; however, it significantly extends the inference time as it requires multiple inferences on each mask. Due to these limitations, we decided not to release the code for this module.

However, if you are interested, I would be more than happy to share the code with you. Could you please let me know your specific usage or interest in that part of the code so I can arrange it accordingly?

Best, Zhening

LilRedWu commented 3 months ago

Hi, I'm interested in the part where you calculate the occlusion rate to refine the mask. Could you please share the related code? That would be very helpful. Thank you again for the outstanding work and the new insights you offer in zero-shot 3D understanding.

ZheningHuang commented 2 months ago

It has been a while, but hopefully, this will still be useful. I presented a demo in

snap_explore.ipynb

which showcases how this is conducted. Let me know if you have more questions

Best, Zhening