search

comfyanonymous / ComfyUI

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
https://www.comfy.org/
GNU General Public License v3.0
57.49k stars 6.09k forks source link

Add strength input to Differential Diffusion #5709

Open EricBCoding opened 16 hours ago

EricBCoding commented 16 hours ago

Hi,

The Differential Diffusion node from #2876 is awesome for inpainting tasks, but the effect is too strong at times. It is challenging to use when generating a subject that has little in common with the original.

Even at a denoising strength of 100%, Differential Diffusion often overwhelms the prompt. But if we blend the adjusted binary mask with the original, we can find a sweet spot. The example below was performed with Euler/Beta, 4 steps, 100% denoising strength. Model is flux1-dev.sft with distillation LoRAs for speed.

differential_diffusion_strength

50-75% Differential Diffusion strength feels a lot more usable in practice than 100%. Here's one more comparison if you'll indulge me:

The prompt was "Donald Trump." That guy in the middle looks like he'd be fun at parties, but he's definitely not Trump.

ltdrdata commented 14 hours ago

Wouldn't it be better to skip the blending logic when the strength is set to 1.0?

EricBCoding commented 14 hours ago

@ltdrdata The blend operation is nearly instantaneous either way (at least at 1024x1024), but you're right, a conditional check is probably smarter. 🙂 Updated the code.