[bug]: Different diffusers SDXL VAE used in Invoke than Auto1111/Forge

[Jonseed]

Is there an existing issue for this problem?

• [X] I have searched the existing issues

Operating system

Windows

GPU vendor

Nvidia (CUDA)

GPU model

RTX 3060

GPU VRAM

12GB

Version number

5.0.0

Browser

Edge 128.0.2739.79 (Official build) (64-bit)

Python dependencies

{ "accelerate": "0.30.1", "compel": "2.0.2", "cuda": "12.1", "diffusers": "0.27.2", "numpy": "1.26.4", "opencv": "4.9.0.80", "onnx": "1.15.0", "pillow": "10.4.0", "python": "3.10.6", "torch": "2.2.2+cu121", "torchvision": "0.17.2+cu121", "transformers": "4.41.1", "xformers": "0.0.25.post1" }

What happened

I'm coming from Auto1111/Forge. When I generate in Invoke, I get a very washed out image. Prompt: "a portrait of a beautiful woman. photography. portraiture. catch light in eyes. one flash. rembrandt lighting. Soft box. dark shadows. High contrast. 80mm lens. F2.8." Negative: "painting, digital art. sketch, blurry"

Here is the very same prompt, model, and all other settings, including scheduler/sampler, steps, cfg, VAE, and seed in Auto1111/Forge:

I understand that the images won't be exactly the same because of slight differences in architecture, but the overall look is quite different here. What might be causing Invoke to be so washed out and low contrast compared to Auto1111/Forge?

What you expected to happen

An image more similar to Auto1111/Forge. I'm not sure what it causing the low contrast washed out look in Invoke.

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

[ufuksarp]

I uploaded the image into my Invoke and recalled all the settings except the model, which I used RealVisXL V3 for that.

This is what I got:

[hipsterusername]

Make sure that you are using the Fixed SDXL VAE.

Else, share all settings used in the generation tab, not just prompt.

[Jonseed]

@hipsterusername thanks. Apparently the VAE was the problem. I was using the fixed SDXL fp16 VAE from madebyollin (sdxl.vae.safetensors), but maybe Invoke doesn't like that one? I downloaded the one from the starter model pack in Invoke (diffusion_pytorch_model.safetensors and .bin, looks like Invoke downloads both extensions), and that one works. Is one for diffusers and the other is a different architecture? This might be something to take special note for users coming over from Auto1111, that they need a different SDXL VAE for Invoke (probably because Invoke uses the diffusers pipeline?), otherwise they are going to get very poor quality images, and it's a bit hard to hunt down the culprit.

With the proper diffusers VAE, now the image generates very well:

[Jonseed]

I guess this issue is solved now, but maybe documentation needs to be added for users transitioning from Auto1111/Forge?

[Jonseed]

Or better yet, perhaps instead of (or in addition to) documentation, when the user has a non-diffusers VAE selected in the UI (if that really is the cause of the issue), then maybe Invoke could detect that and it could highlight or outline the VAE selector box yellow and give a warning that the selected VAE may not work as intended or give suboptimal results because of an incompatible format, and recommend that the user download the VAE from the starter models in the model manager. Or Invoke may simply refuse to use the VAE because it is in an incompatible format?