search

facebookresearch / segment-anything

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
Apache License 2.0
47.68k stars 5.64k forks source link

the model is really slow #420

Open msverma101 opened 1 year ago

msverma101 commented 1 year ago

i used the following setup and it is really slow to generate the mask it took almost 8 minutes

vit_b, sam_vit_b_01ec64.pth and a simple image of size 5.9mb

can you tell me what i can do to reduce this time

lavishchauhan321 commented 1 year ago

Encoder is pretty large and wont run on any cpu, try using gpu

lavishchauhan321 commented 1 year ago

yeah automatic mask generator is pretty slow, there are few things you can try 1) resize your image to a smaller image and then resize the masks that you got back to original image 2) use smaller model 3) try manually passing 32x32 = 1024 points in batches of say 128 or 256, you will get masks pretty quickly using this. After that remove duplicate / near duplicate masks, put iou thresholding, in my case all this look 6s for 3000 x 3000 images. And the results were more than satisfactory in my case. 3rd point will be particularly helpful in you are trying to segment particular object/ objects in your image

ByungKwanLee commented 1 year ago

https://github.com/ByungKwanLee/Full-Segment-Anything addresses the ciritical issues of SAM, which supports batch-input on the full-grid prompt (automatic mask generation) with post-processing: removing duplicated or small regions and holes, under flexible input image size

Jordan-Pierce commented 1 year ago

yeah automatic mask generator is pretty slow, there are few things you can try

  1. resize your image to a smaller image and then resize the masks that you got back to original image
  2. use smaller model
  3. try manually passing 32x32 = 1024 points in batches of say 128 or 256, you will get masks pretty quickly using this. After that remove duplicate / near duplicate masks, put iou thresholding, in my case all this look 6s for 3000 x 3000 images. And the results were more than satisfactory in my case. 3rd point will be particularly helpful in you are trying to segment particular object/ objects in your image

@lavishchauhan321 would you mind elaborating on this a bit more? Do you use the built-in tools for thresholding or something like OpenCV? Also, do you happen to know what bottleneck you're removing in the Generator tool when switching to a Generator-like tool via the Predictor tool that allows for such an increase in speed?