sail-sg / EditAnything

Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)
Apache License 2.0
3.23k stars 179 forks source link

How to use the trained model by myself? #45

Open lyc6749 opened 1 year ago

lyc6749 commented 1 year ago

I have trained the model using sam_train_sd21.py,please tell me how to use the trained model for inference.

gasvn commented 1 year ago

use this to transfer the weights to diffusers style: https://github.com/sail-sg/EditAnything/blob/main/tools/convert_controlnet_to_diffusers.py

lyc6749 commented 1 year ago

when I used the convert model for test, run with editany_test.py,I meet the problem image

lyc6749 commented 1 year ago

when I used the convert model for test, run with editany_test.py,I meet the problem image

maybe only support for sd1.5?

gasvn commented 1 year ago

The default training code is for sd2.1, you need to use sd2.1 base model for your trained controlnet. Or you can train a sd1.5 based version.

lyc6749 commented 1 year ago

I changed the code of editany_test.py , but the error is still appeared. image

lyc6749 commented 1 year ago

image

gasvn commented 1 year ago

I changed the code of editany_test.py , but the error is still appeared. image

Change extra_inpaint=False, for SD2.1, we use the SD-inpaint model, so no extra inpaint model is needed. The extra inpaint model is the controlnet model only suitable for SD1.5