sail-sg / EditAnything

Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)
Apache License 2.0
3.34k stars 193 forks source link

where is the file sam2edit.py,seems not exist #49

Closed SkyCol closed 1 year ago

SkyCol commented 1 year ago

Hello, I want to edit image and don't change the mask rigion. If without any UI, which python file should I read. I just change sam2Image.py by inputing a single point to segment-anything model, but the rigion still changed(as I want to use like inpaint). Can you teach me about this , thank you!

gasvn commented 1 year ago

Hi, the sam2edit is renamed to editany. The class EditAnythingLoraModel can be used without UI. If you want to keep the unmasked region, you need to set the ‘mask_image’

SkyCol commented 1 year ago

Thank you! It shows inpaint can only used with SD1.5 , can I use sd2.1 to do this?

gasvn commented 1 year ago

You can use sd2.1 by the following way. Due to the SD2.1 uses different inpainting method, it cannot support lora model for now. model = EditAnythingLoraModel( base_model_path="stabilityai/stable-diffusion-2", controlmodel_name="LAION Pretrained(v0-4)-SD21", extra_inpaint=False, lora_model_path=None, use_blip=True)

SkyCol commented 1 year ago

ok,but when using sd1.5 and LAION Pretrained SD1.5,I get an error: RuntimeError: mat1 and mat2 shapes cannot be multiplied (66x768 and 1024x320)_ and when I use sd2.1 and LAION Pretrained SD2.1(extra_inpaint = false) , I get a result like this(want to redraw the car)

image

By the way ,What's the parameter extra_inpaint means. thankyou for your reply.

SkyCol commented 1 year ago

My target is to replace the car according to my prompt(without blip), here is the result on your website ,it‘s great , What should I do to achieve this?I call model.procee but I failed.

image
SkyCol commented 1 year ago

Sorry,I tried successfully on SD1.5,The error I mentioned RuntimeError: mat1 and mat2 shapes cannot be multiplied (66x768 and 1024x320)_ is because I choose a wrong path of controlNet , 768 is embedding size of SD1.5 while 1024 is SD2.1. It's my mistake..

affromero commented 1 year ago

@SkyCol how/where exactly did you fix the RuntimeError?