NielsRogge / Transformers-Tutorials

This repository contains demos I made with the Transformers library by HuggingFace.
MIT License
8.51k stars 1.34k forks source link

How to extract masked image? by using CLIPSeg or any better alternatives? #252

Open FurkanGozukara opened 1 year ago

FurkanGozukara commented 1 year ago

Hello. Thank you so much for all amazing work

I am looking for a way to extract clothes from given picture

I will show a raw example

Let's say this is our given image

image

And this is the output that I am looking for

image

I tested this but it shows only masked part : https://huggingface.co/spaces/nielsr/CLIPSeg

So how can I achieve this?

Is CLIPSeg the best thing to do this? any alternatives? thank you so much

I used clothes prompt any other prompts you suggest?

NielsRogge commented 1 year ago

Hi,

Yes you first use CLIPSeg to come up with a binary mask (as shown in my example notebook), and then crop out the part of the image based on the mask as explained here.

I'd recommend trying out a few different text prompts and see which one works best. You can also provide an image prompt instead of a text prompt as explained in our blog post.