Also called "guess_mode" in diffusers. Implemented via the uncond_multiplier on Soft Weights. When set to 0.0, replicates auto1111 exactly, but any value between 0.0 and 1.0 also works without issue, allowing for granular control.
In terms of how it works under the hood, "ControlNet is more important" makes the control input/output not modify the uncond, so in auto1111 it simply multiplies the uncond controlnet components by 0.0. No reason why you couldn't make this granular, so I exposed it as uncond_multiplier instead of a boolean.
In ComfyUI, the controlnet components do not get direct access to know where the conds and unconds are located in the passed-in latents, so for now I take advantage of the fact that the built-in len function is used when calling get_control in calc_conds_batch, and prior to that, comfy.samplers.cond_cat is called ONLY in calc_conds_batch. Assuming all other custom calc_conds_batch implementations also keep this order of execution (and they should have to), I make cond_cat wrap the len function so that when the object that len is being taken care of matches the characteristics of the cond_or_uncond list (contains 1's and 0's), it will instead return a IntWithCondOrUncond object containing the (alleged) cond_or_uncond value, then it will clean itself up and restore itself to the original len function. This prevents things from going awry if other threads use the len function between the cond_cat wrapping and the len function unwrapping. Using this hacked int object, get_control can then access cond_or_uncond for further use. An additional wrapper around the default sampling functions (for ksampler/custom ksampler) to not use the wrapped cond_cat function if no controlnet weights have a non-default uncond_multiplier. From testing, there is only the small fraction of a percent speed difference between doing cond_cat being wrapped all the time vs not, but I will do this just in case.
The advantage of this hacky approach vs making separate ControlNets for cond vs uncond is that this will work with Timestep Keyframes w/ different weights out of the box.
As soon as ComfyUI makes controlnet stuff have access to cond_or_uncond, I will refactor this code to no longer require these wraps
Also called "guess_mode" in diffusers. Implemented via the uncond_multiplier on Soft Weights. When set to 0.0, replicates auto1111 exactly, but any value between 0.0 and 1.0 also works without issue, allowing for granular control.
In terms of how it works under the hood, "ControlNet is more important" makes the control input/output not modify the uncond, so in auto1111 it simply multiplies the uncond controlnet components by 0.0. No reason why you couldn't make this granular, so I exposed it as uncond_multiplier instead of a boolean.
In ComfyUI, the controlnet components do not get direct access to know where the conds and unconds are located in the passed-in latents, so for now I take advantage of the fact that the built-in len function is used when calling get_control in calc_conds_batch, and prior to that, comfy.samplers.cond_cat is called ONLY in calc_conds_batch. Assuming all other custom calc_conds_batch implementations also keep this order of execution (and they should have to), I make cond_cat wrap the len function so that when the object that len is being taken care of matches the characteristics of the cond_or_uncond list (contains 1's and 0's), it will instead return a IntWithCondOrUncond object containing the (alleged) cond_or_uncond value, then it will clean itself up and restore itself to the original len function. This prevents things from going awry if other threads use the len function between the cond_cat wrapping and the len function unwrapping. Using this hacked int object, get_control can then access cond_or_uncond for further use. An additional wrapper around the default sampling functions (for ksampler/custom ksampler) to not use the wrapped cond_cat function if no controlnet weights have a non-default uncond_multiplier. From testing, there is only the small fraction of a percent speed difference between doing cond_cat being wrapped all the time vs not, but I will do this just in case.
The advantage of this hacky approach vs making separate ControlNets for cond vs uncond is that this will work with Timestep Keyframes w/ different weights out of the box.
As soon as ComfyUI makes controlnet stuff have access to cond_or_uncond, I will refactor this code to no longer require these wraps