search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
47.87k stars 5.66k forks source link

Performance Discrepancy Between SAM Model Demo and GitHub Code #762

Open i-am-invincible opened 5 months ago

i-am-invincible commented 5 months ago

Hi, I am working on segmenting car bodies in images using the Meta SAM model. I am facing a significant difference in performance between the UI demo on the official website and the code provided on the GitHub repository. UI demo performed remarkably well with just 1-2 clicks, however, when I attempted to use the code, results are very different and bad. Despite of providing multiple points, the results were not up to the mark as compared to the demo.

Using SAM Model Version:- "vit_h" Used predictor_example file:- notebooks/predictor_example.ipynb

Examples: Image 1: Original Image: image3

UI Demo Segmentation: - Performed well with 4 foreground points and 3 background points. resized_sam_ui_3

My Code Segmentation: - Poor results with the same point placement. code_output_3

Image 2: Original Image: image2

UI Demo Segmentation: - Good results with 4 foreground points and 4 background points. resized_sam_ui_2

My Code Segmentation: - Poor results with the same point placement. code_output_2

I would appreciate any insights into why this discrepancy is happening. Could it be related to hidden hyperparameter settings, optimizers, or learning rates used in the UI demo that aren't included in the GitHub code? If this is the case, would it be possible to provide some guidance.

zhywyt commented 2 months ago

I have the same question.

M3LLI55X commented 2 months ago

same problem

scchess commented 2 months ago

Same issue.

facundoq commented 2 months ago

The online and library versions are probably using different hyperparamters. Checkout this notebook: https://github.com/facebookresearch/segment-anything/blob/main/notebooks/automatic_mask_generator_example.ipynb

scchess commented 2 months ago

What exactly to check in the notebook?

facundoq commented 2 months ago

Cell 24 indicates the config options. The parameter names are mostly self explanatory.

El mar, 17 de sept de 2024, 15:36, Ted Wong @.***> escribió:

What exactly to check in the notebook?

— Reply to this email directly, view it on GitHub https://github.com/facebookresearch/segment-anything/issues/762#issuecomment-2356635060, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMA6BZXPR7CVK7JPAT2Z6DZXBZD3AVCNFSM6AAAAABJPURO5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJWGYZTKMBWGA . You are receiving this because you commented.Message ID: @.***>

zhchlong commented 2 months ago

same question.