Replace the content of the mask with an image?

[lunatico67]

The normal way is to create a mask and write a prompt to modify its content. What I want is to modify the content of that mask with the content of an image, not with a prompt, for example, I create a mask of a woman's shirt and I want to change it for a texture of an image. I guess it is possible with ControlNet inpaint but I don't know how to do it.

[Uminosachi]

Please download the IP adapter model for ControlNet from the links below and save it in the 'extensions/sd-webui-controlnet/models/' folder. Then, restart the webUI. Doing so will enable the use of the IP adapter in the 'ControlNet options' found under the 'ControlNet Inpaint' tab. The IP adapter becomes active when you drag and drop a reference image into the 'Reference Image' area.

• https://huggingface.co/lllyasviel/sd_control_collection/resolve/main/ip-adapter_sd15.pth?download=true
• https://huggingface.co/lllyasviel/sd_control_collection/resolve/main/ip-adapter_sd15_plus.pth?download=true

Before you begin, make sure to set 'Multi-ControlNet' to 2 or higher in the 'ControlNet' section of the webUI settings. Additionally, when using the 'Reference-Only' mode in the 'ControlNet options', you can use this feature without the need to download any models.

[lunatico67]

Thanks for the help.

I must have misconfigured something because nothing comes out in the resulting image.

[Uminosachi]

Please select the SD 1.5 model from the 'Stable Diffusion checkpoint' list in the top left of the webUI. It does not support other models such as SDXL.

[lunatico67]

Thanks! Works with Reference only, with IPAdapter the resulting shirt has nothing to do with the reference picture