Support multiple control nets in the `StableDiffusionControlNetXSPipeline`/`StableDiffusionXLControlNetXSPipeline`

[chrismaltais]

Is your feature request related to a problem? Please describe. The StableDiffusionControlNetPipeline/StableDiffusionXLControlNetPipeline allow you to specify a list of control nets (i.e. depth and canny)

However, the new ControlNetXS pipelines only allow you provide a single control net

Describe the solution you'd like. I want to be able to do:

vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
canny_controlnet = ControlNetXSAdapter.from_pretrained(
    "UmerHA/Testing-ConrolNetXS-SDXL-canny", torch_dtype=torch.float16
)
depth_controlnet = ControlNetXSAdapter.from_pretrained(
    "UmerHA/Testing-ConrolNetXS-SDXL-depth", torch_dtype=torch.float16
)
pipe = StableDiffusionXLControlNetXSPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", controlnet=[canny_controlnet, depth_controlnet], torch_dtype=torch.float16
)

[sayakpaul]

Thanks for your interest. Would you want to open a PR to add this support? We'd be more than happy to guide you. Cc: @UmerHA as well.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[KarthikSundar2002]

I would like to work in this issue, and this is my first time contributing to Diffusers, so would really like some guidelines on where to start

[sayakpaul]

Thanks for that. Feel free to start working on it and ask questions along the way.