search

facebookresearch / segment-anything-2

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
11.08k stars 940 forks source link

Prompting with vague segmentation mask #327

Open mano3-1 opened 4 days ago

mano3-1 commented 4 days ago

Hi,

I tried prompting the model with a vague segmentation mask that I obtained from another model. The prompt isn’t very accurate, but it at least covers the object of interest. I expected SAM2 to refine the mask more precisely, but the results are not better.

Here is the mask prompt: image

And result from SAM: image

There are small patches scattered on the right side, and the left side of the mask is completely off.

What could be the reason? Does SAM2 always need a point/box to refine the masks?

Caspeerrr commented 1 day ago

One thing you could try is transforming the segmentation mask to a bounding box and using that as input to sam2: torchvision.ops.masks_to_boxes