search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
45.59k stars 5.38k forks source link

Image embedding using own backbone model are of lesser quality than demo #224

Open OmerMachluf opened 1 year ago

OmerMachluf commented 1 year ago

Hi and thanks for sharing this fabulous work! I've been trying to repro the results from the demo. It seems that the resulted image embedding that returns from the backbone model at modelzoo is different than the one returned when hosting my own backbone model. I'm hosting vit h model. When using these two different embedding with onnex and gui, the differences becomes very apparent. It seems like the modelzoo returns a much superior result for the embedding. When trying to select some areas using gui I'm getting a lot of missing data/holes for my embedding where things just work perfectly with the model hosted in modelzoo Would appreciate your help with understanding:

  1. Which version of model is hosted in model zoo
  2. Is the inference code for getting an embedding any different or contains some confit edits to obtain those results
  3. Any other factors you can think of that could contribute to obtaining the superior result Much thanks about this work!
wkkuniquegmail commented 1 year ago

same problem.

zhaoweiyu-github commented 1 year ago

It seems that the resulted image embedding that returns from the backbone model at modelzoo is different than the one returned when hosting my own backbone model. I'm hosting vit h model.

Hi, would you like to provide a link for the mentioned backbone model at model zoo? I cannot find it. Thanks!

OmerMachluf commented 1 year ago

It seems that the resulted image embedding that returns from the backbone model at modelzoo is different than the one returned when hosting my own backbone model. I'm hosting vit h model.

Hi, would you like to provide a link for the mentioned backbone model at model zoo? I cannot find it. Thanks!

Is this still relevant?. you can find it in the outoging network requests in the developer tools, it's two requests sent to modelzoo when you upload an image(and embedding does not yet exist). one to get the image embedding, the other to get masks

helen1c commented 1 year ago

Same problem here. @ericmintun can you just tell us if you're using a different model or other hyperparameters that shouldn't be shared?

Muccul commented 1 year ago

+1

smandava98 commented 6 months ago

Hi @nikhilaravi @HannaMao can we please get a clarification on this?