Is it possible to make the inference a bit more faster? - Githubissues

Curt-Park / segment-anything-with-clip

Segment Anything combined with CLIP

https://huggingface.co/spaces/curt-park/segment-anything-with-clip

Apache License 2.0

328 stars 23 forks source link

Is it possible to make the inference a bit more faster? #2

Open CriusFission opened 1 year ago

CriusFission commented 1 year ago

Hi, @Curt-Park, thanks for sharing your work. I'm trying to run this on a bunch of images in a folder. It contains around 100 images. It takes a few hours on my RTX 3070. Is it possible to make it a bit faster?

Curt-Park commented 1 year ago

The execution speed is heavily dependent on the make generator algorithm in Segment Anything, which runs 1024 predictions per a single image.

We have two solutions:

Use a smaller model.
Enhance the performance of SAM's mask generator.

CriusFission commented 1 year ago

Thanks for the reply. Using a smaller model is helpful. I apologize if the question is silly, but how do I go about enhancing the performance of SAM's mask generator?

Curt-Park commented 1 year ago

I made a new algorithm that runs fast on CPU. However, I can not open it because I made it for work. I am sorry for that.

DEMO: https://youtu.be/y9AAPsTCW3I

CriusFission commented 1 year ago

Looks cool! Is it possible to do batched inferencing with this model? As I'm having about 100 images, it would make sense to provide the input as a batch.

Curt-Park commented 1 year ago

@CriusFission it already does 64 batched inferences at once. FYI: https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py#L40

Kirang96 commented 1 year ago

I meant running multiple images at a time in different cores.

Curt-Park commented 1 year ago

I meant running multiple images at a time in different cores.

As for this repository, you can run multiple gradio apps at once.

Curt-Park commented 1 year ago

I made a new algorithm that runs fast on CPU. However, I can not open it because I made it for work. I am sorry for that.

FYI, we opened the demo pages on hugging face spaces!

Fast Segment Everything: Re-implemented Everything algorithm in iterative manner that is better for CPU only environments. It shows comparable results to the original Everything within 1/5 number of inferences (e.g. 1024 vs 200), and it takes under 10 seconds to search for masks on a CPU upgrade instance (8 vCPU, 32GB RAM) of Huggingface space.
Fast Segment Everything with Text Prompt: This example based on Fast-Segment-Everything provides a text prompt that generates an attention map for the area you want to focus on.
Fast Segment Everything with Image Prompt: This example based on Fast-Segment-Everything provides an image prompt that generates an attention map for the area you want to focus on.
Fast Segment Everything with Drawing Prompt: This example based on Fast-Segment-Everything provides a drawing prompt that generates an attention map for the area you want to focus on.