search

nv-nguyen / cnos

[ICCV 2023 R6D] PyTorch implementation of CNOS: A Strong Baseline for CAD-based Novel Object Segmentation based on Segmenting Anything and DINOv2
MIT License
204 stars 22 forks source link

Training Details #1

Closed omega-cw closed 1 year ago

omega-cw commented 1 year ago

Hi, thanks for your nice work! And I have some questions hope you can help me:

  1. during training of proposal stage, the usage of SAM(or Fast SAM) is promptable or not promptable. If promptable, which type of prompt do you use (point or bbox), and how do you get them? And if not promptable, how do deal with the over-segment problem.
nv-nguyen commented 1 year ago

To be clear, there is no training in our framework. During the onboarding and proposal stage, we use the pretrained model provided by SAM (or FastSAM) and DINOv2.

For SAM (or FastSAM), we use its default not prompt setting by sampling 64 points per dimension. There is still a problem of over-segmentation but it can be filtered in the matching stage since their over-segmentation masks have lower confidence score comparing to correct segmentation masks. For example, we show in the teaser the segmentation on YCB-V dataset and most of the over-segmentation masks are filtered out.