search

mikonvergence / ControlNetInpaint

Inpaint images with ControlNet
MIT License
342 stars 29 forks source link

MultiControlNet support? #6

Open oxysoft opened 1 year ago

oxysoft commented 1 year ago

In the original ControlNet pipeline, we can pass a list of controlnet models like this

        self.ptxt = StableDiffusionControlNetPipeline.from_pretrained(
                "runwayml/stable-diffusion-v1-5",
                safety_checker=None,
                requires_safety_checker=False,
                controlnet=[
                    ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-canny", torch_dtype=torch.float16),
                    ControlNetModel.from_pretrained("lllyasviel/sd-controlnet-depth", torch_dtype=torch.float16)
                ],
                torch_dtype=torch.float16).to("cuda")

Is this supported in this pipeline?

Cheers

mikonvergence commented 1 year ago

I believe that's a feature introduced in diffusers==0.15.0?

It will be possible to do that, but there are currently still some interface inconsistencies with the latest version of diffusers (as mentioned in https://github.com/mikonvergence/ControlNetInpaint/pull/7).

Once that's sorted out, I will also make sure that multi-ControlNet approach is also possible within this tool.

FeiyanWu9 commented 2 months ago

hi，Can ControlNet-Inpaint be used in combination with other ControlNets now?

mikonvergence commented 2 months ago

Hi @FeiyanWu9 - unfortunately I haven't had time to work on this and won't be able to do it in the near future. Anyone is welcome to contribute though, as long as it stays within the original environment versions (upgrading the codebase to a newer version of diffusers is not foreseen).