search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
26.17k stars 5.39k forks source link

add skip layer guidance to sd 3.5 #9819

Open vladmandic opened 1 week ago

vladmandic commented 1 week ago

stabilityai recently added support to skip_layer_guidance in their official implementation via this pr: https://github.com/Stability-AI/sd3.5/pull/11/files

slg adds extra guidance to the original cfg during specific steps. its calculated as positive output from original model minus positive model output from a model with specific layers removed (e.g. layer 7, 8 and 9 for sd35-medium).

slg is an optional feature, but seems like sai prefers that its enabled for sd35 medium variant.

cc: @yiyixuxu @sayakpaul @DN6 @asomoza

sayakpaul commented 1 week ago

In the works.

bghira commented 5 days ago

well since it's been a week without movement i opened #9880 to cover this base though it doesn't apply for img2img or other things yet. those can be added after.