lshqqytiger / stable-diffusion-webui-amdgpu

Stable Diffusion web UI
GNU Affero General Public License v3.0
1.8k stars 186 forks source link

Model ControlNet ONNX usable on Stable Diffusion WebUI DirectML #346

Open DIKODEY opened 9 months ago

DIKODEY commented 9 months ago

Is there an existing issue for this?

What would your feature do ?

Hello, I would like to know if it's possible to add a feature to use an ONNX model of ControlNet directly on the stablediffusion interface. The model converted via /Amblyopius/Stable-Diffusion-ONNX-FP16 works, but not with the StableDiffusion interface:

raise ValueError( ValueError: Required inputs (['down_block_0', 'down_block_1', 'down_block_2', 'down_block_3', 'down_block_4', 'down_block_5', 'down_block_6', 'down_block_7', 'down_block_8', 'down_block_9', 'down_block_10', 'down_block_11', 'mid_block_additional_residual']) are missing from the input feed (['sample', 'timestep', 'encoder_hidden_states']).

Proposed workflow

  1. Creation of an OnnxControlnet file in the models folde 2.Creation of the interface for Stable Diffusion UI DirectML as shown in the image below. 3.When the StableDiffusion checkpoint matches with ControlNet, image generation becomes possible. 4.The benefit would be to increase the execution speed when ControlNet is used.

Capture d'écran 2023-12-30 213805

Additional information

No response