Mask Label

[bsridatta]

Since FastSAM uses Yolo detector, is it possible to get the mask labels?

[zxDeepDiver]

FastSAM builds upon the YOLOv8X-seg algorithm. It utilizes segmentation masks obtained from SA-1B.

[bsridatta]

Thanks for the reply. https://docs.ultralytics.com/tasks/segment/ Here you can see the description of Yolov8-seg "The output of an instance segmentation model is a set of masks or contours that outline each object in the image, along with class labels and confidence scores for each object. Instance segmentation is useful when you need to know not only where objects are in an image, but also what their exact shape is."

so is there a way to get labels for the masks?

[zxDeepDiver]

This is an interesting topic for further study. As we currently train on SA-1B, FastSAM is unable to provide class labels for the generated masks. However, there are several engineering techniques that could be explored:

1. Modifying the CLIP model in text-prompt mode to act as a zero-shot classifier to obtain the labels.
2. Running a detector to obtain bounding boxes with labels, which can then be used with the box-prompt mode of FastSAM to generate masks.