Discrepancy results between online demo and SamAutomaticMaskGenerator

[michelbotros]

Hi,

I have been trying out SAM in the online demo and it works great using the "Everything" button:

Image:

Result demo:

I then tried to recreate these results with the code using the SamAutomaticMaskGenerator and a vit-h (default) model. See code:

# open the image
image = cv2.imread('image.png')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# initialize SAM vit-h and a mask_generator
sam_vit_h = sam_model_registry['vit_h'](checkpoint='checkpoints/sam_vit_h_4b8939.pth').to(device='cuda')
mask_generator_vit_h = SamAutomaticMaskGenerator(sam_vit_h)

# generate masks
masks = mask_generator_vit_h.generate(image)

# plot
plt.figure(figsize=(10, 10))
plt.imshow(image)
show_anns(masks, one_color=False)
plt.axis('off')
plt.savefig('result_code.png', bbox_inches='tight', pad_inches=0)
plt.show()

However,` I get a different result. Large portion of the background is segmented and there is also many more small regions detected.

Result code:

For my application the demo result is preferred. If anyone knows how to recreate the demo results, it would be greatly appreciated if its shared!

[MichaelP84]

I am experiencing the same thing, just between the online demo and the workflow showed in predictor_example.ipynb on my own images

[scchess]

Yeah same here

[Nivratti]

The primary reason for the variation in masks between the online demo and the local notebook is that the online demo uses a different model for "segment everything" functionality. The online demo is powered by the model hosted at the following endpoint:

segment_everything_box_model

[CurryKd7]

在线演示和本地笔记本之间掩码差异的主要原因是，在线演示使用不同的模型来实现“分割所有内容”功能。在线演示由托管在以下终端节点的模型提供支持：

segment_everything_box_model

The link you provided cannot be opened, why

[pzhren]

Indeed, the effect of online demo is obviously better than local, why?

[scchess]

There is no such thing as free lunch, the best tuned model is of course unavailable.

[Nivratti]

在线演示和本地笔记本之间掩码差异的主要原因是，在线演示使用不同的模型来实现“分割所有内容”功能。在线演示由托管在以下终端节点的模型提供支持： segment_everything_box_model

The link you provided cannot be opened, why

The link I provided is an API reference URL that I identified by inspecting and analyzing the web application's code. The online demo model named segment_everything_box_model is hosted on private servers of meta and isn't publicly accessible. Online demos interact with this private model via API calls.