Add Perturbed Attention Guidance

[v0xie]

WIP implementation of https://arxiv.org/abs/2403.17377

TODO:

• Enable switch between "PAG only", "PAG + CFG"
• Add slider for "Use CFG for only a percent of steps"
• Add control for choosing layers to perturb
• ~For some reason the last value for PAG Scale persists between iterations unless disabled then re-enabled.~
  • PAG scale persisting is still a problem, so we're using a global variable to manage the state (pending fix in upstream PR https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15428)
• XYZ Plot Options
• ~Refactor to clean up the messy code~
• ~Only works for SD 1.5 (unsure of layers to perturb for SD XL since paper only uses Stable Diffusion 1.5)~

[v0xie]

Still grappling with the pag_scale parameter persisting between jobs. Temporary solution is to use a global.

both PAG+CFG

v1-5-pruned-emaonly.safetensors [6ce0161689]

Aidv210AnimeIllustDiffusion_aidv210.safetensors [4efd47fad9]

[v0xie]

Major cleanup and a test implementation for SD XL.

I didn't see anything in the paper about SD XL so I'd be interested to see ablation results for perturbing different layers. Currently perturbing the same layer as recommended for SD 1.5.

Interesting results when negating the guidance scale. For some seeds/prompts, the negative guidance scale produces something subjectively better than the positive guidance scale does.

sd_xl_base_1.0_0.9vae.safetensors [e6bb9ea85b]