How is it different from multi-controlnet?

[i-amgeek]

Architecture might be different but I am having trouble understanding if this is different from multi-controlnet for application purpose? Does it enable any new functionality or does it improve resultant image quality as compared to multi-controlnet?

[canqin001]

Hi i-amgeek,

That's an excellent question. The primary distinction lies in the model's size and configuration. Our UniControl operates as a single, comprehensive model (1.4B parameters, 5.78GB checkpoint, https://console.cloud.google.com/storage/browser/_details/sfr-unicontrol-data-research/unicontrol.ckpt) that can handle all tasks concurrently. In contrast, the Multi-ControlNet requires the coordination of multiple task-specific ControlNets (SD+N ControlNets, https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main) to function properly, each taking up 361M parameters and 1.45GB storage, which results in a heavier, more challenging model to deploy (SD also takes 1.1B parameters).

We believe that a compact, unified model such as UniControl represents a promising direction for the field. Moreover, we anticipate that our unified approach will provide additional benefits, exceeding the performance of a single-task ControlNet. Our user studies substantiate these expectations with positive results. We eagerly await further user feedback to continue enhancing our system.

Thank you!

[2blackbar]

Can You guys extract the model so it can be used with our own custom models ? i guess asking for extension for auto11 webui is too much , maybe there will be someone who can adapt it

[canqin001]

Yes. Thank you for your suggestion. We will extract model to let developers to easily deploy UniControl on their own dataset. I will likely update it next week.

[2blackbar]

Great news