search

AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
135.2k stars 25.82k forks source link

[Feature Request]: Support for SDXL inpainting models (9 channel) #13055

Closed George0726 closed 7 months ago

George0726 commented 10 months ago

Is there an existing issue for this?

What would your feature do ?

Support for SDXL-inpainting models.

Proposed workflow

  1. The difference between SDXL and SDXL-inpainting is that SDXL-inpainting has an additional 5 channel inputs for the latent feature of masked images and the mask.
  2. I have tried to modify by myself but there seem like some bugs image
  3. I modified the config.yaml in repositories/generative-models/configs/inference to support for 9 channel input and modified the repositories/generative-models/sgm/modules/diffusionmodules/wrappers.py to concat with additional 5 channels x = torch.cat([x, c['c_concat'][0]],dim=1) in line29.
  4. It would be helpful if you can support for the current SDXL inpainting models.

Additional information

No response

Cleroth commented 10 months ago

Title of the issue is blank. Someone apparently got it working with automatic1111 here, though I haven't tried all those steps.

George0726 commented 10 months ago

It seems like they directly use diffusers pipeline.