[Issue]: Washed out look in images after updating to diffusers 0.26

[lbeltrame]

Issue Description

When generating images with diffusers 0.26 and SDXL (at least with Animagine XL v.3) the images will have a distinct washed-out look, as if a grey filter has been applied over. Downgrading to diffusers 0.25.1 makes the problem go away, but it's obviously a non viable solution.

Reproduction information

Model: Animagine XL 3 VAE: madebyollin/sdxl-vae-fp16-fix

Prompt: 1girl, standing, hands on hips, smile anime artwork, anime style, key visual, vibrant, studio anime, highly detailed, anime coloring, newest, masterpiece, best quality Negative: lowres, worst quality, low quality, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, photo, deformed, black and white, realism, disfigured, low contrast

These were reproduced on a fresh install of SD.Next on a different platform than I use usually.

The metadata are in the two images attached. As I strongly suspect it's configuration dependent, I'm also attaching the config.json from the instance I used.

Images

With diffusers 0.26.2

With diffusers 0.25.1

Configuration

Config.json config.json

Version Platform Description

16:25:23-028323 INFO Starting SD.Next
16:25:23-038324 INFO Logger: file="/workspace/automatic/sdnext.log" level=INFO size=155241 mode=append
16:25:23-045151 INFO Python 3.10.12 on Linux
16:25:23-516021 INFO Version: app=sd.next updated=2024-02-08 hash=376bace0 url=https://github.com/vladmandic/automatic/tree/master
16:25:24-723396 INFO Platform: arch=x86_64 cpu=x86_64 system=Linux release=5.4.0-159-generic python=3.10.12

Relevant log output

Not applicable.

Backend

Diffusers

Branch

Master

Model

SD-XL

Acknowledgements

• [X] I have read the above and searched for existing issues
• [X] I confirm that this is classified correctly and its not an extension issue

[vladmandic]

big rule of troubleshooting is always try to simplifiy first.

look at metadata:

info: namespace(Prompt='1girl, standing, hands on hips, smile  anime artwork, anime style, key visual, vibrant, studio anime,  highly detailed, anime coloring, newest, masterpiece, best quality', Negative prompt='lowres, worst quality, low quality, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped,
worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name, photo, deformed, black and white, realism, disfigured, low contrast', Steps=28, Seed=1399088579, Sampler='Euler a', CFG scale=7, Size='1216x832', Parser='Full parser', Model='animagine-xl-3.0', Model hash='1449e5b0b9',
VAE='sdxl.vae', Clip skip=2, Backend='Diffusers', App='SD.Next', Version='376bace', Operations='upscale; hires; txt2img', Second pass='True', Hires force='True', Hires steps=20, Hires upscaler='RealESRGAN 4x+ Anime6B', Hires upscale=2, Hires resize='0x0', Hires size='2432x1664', Denoising strength='0.5', Hires sampler='Euler a', Image CFG
scale=7, CFG rescale=1, Sampler options='', Pipeline='StableDiffusionXLPipeline', Sampler Eta='0.667')

why external vae? is upscaling or hires relevant here? clip skip? eta? non-standard size? etc...

try to reproduce with nothing but basic prompt. if you cant, then gradually increase complexity of params until its reproducible.

i'm sorry i have to push back on this until its clearer. i just cant work with this as-is.

[Disty0]

Reproduced this when using an external VAE.

Animagine has the same VAE baked in too, so this is not the VAE's problem. I am suspecting the config not being loaded properly.

Baked in:

Same VAE but loaded as external:

[Disty0]

Confirmed it by hard coding SDXL VAE's scaling_factor (0.13025) to full_vae_decode and it fixes this. Wrong config is being used somehow.

[vladmandic]

fixed upstream via https://github.com/huggingface/diffusers/pull/6950