[bug]: black image when using IP adapter after updating developer install

[hsm207]

Is there an existing issue for this problem?

• [X] I have searched the existing issues

Operating system

macOS

GPU vendor

Apple Silicon (MPS)

GPU model

No response

GPU VRAM

No response

Version number

4.2.4

Browser

Google Chrome version 126.0.6478.116 (Official Build) (arm64)

Python dependencies

{ "accelerate": "0.30.1", "compel": "2.0.2", "cuda": null, "diffusers": "0.27.2", "numpy": "1.26.4", "opencv": "4.9.0.80", "onnx": "1.15.0", "pillow": "10.3.0", "python": "3.11.9", "torch": "2.2.2", "torchvision": "0.17.2", "transformers": "4.41.1", "xformers": null }

What happened

Always get black image when IP adapter is turned on

What you expected to happen

Get an image that is consistent with the image used in IP adapter

How to reproduce the problem

1. Update fork to commit cd9dfefe3c72419b8908062f45c4edda4d27dc81
2. Generate any image using text to image e.g. a lush green landscape
3. Turn on IP adapter and use the generated image as the control image
4. Generate a new image using text to image e.g. a dry barren landscape
5. Get black image

Additional context

• Used sdxl ip_adapter model ip-adapter-plus-face_sdxl_vit-h for the IP adapter model
• Tried a few SD XL models and different settings for IP adapter in the accordian and all gave the same results.
• No errors or warnings in the logs
• This bug does not happen if we checkout to commit ff01ceae996713dae371ab322db8a32854f0cf34

Discord username

No response

[psychedelicious]

Can you double-check the FP32 toggle in the image to latents node? some SDXL models (like the base) require you to enable FP32, unless you are using a custom VAE that supports FP16. Really annoying but that's how it was released

[hsm207]

Can you double-check the FP32 toggle in the image to latents node?

How do I check that? I've only been working in the Canvas and occasionally the Generation view. It sounds like the image to latents node is a Workflow Editor thing.

On the Canvase, in the Advanced accordian, I have this:

I can't select anything for VAE. It has always been like this. The main sdxl model I'm using is RealitiesEdgeXLLIGHTNING_TURBOV7

[psychedelicious]

Oh, sorry. Yes, I was thinking nodes - it's that VAE Precision setting in the canvas/generation view. FP32 is the most compatible mode, it should work.

I'm attempting to reproduce this issue:

• JuggernautXL (v8 I think?)
• Same IP adapter as you, downloaded using this HF repo id InvokeAI/ip-adapter-plus-face_sdxl_vit-h
• Generate on canvas
• Enable that IP adapter
• Generate again

It works for me.

I think this is the model you are using: https://civitai.com/models/129666/realities-edge-xl-lightning-turbo

It's going to take me hours to download it - civitai is running very slowly for me. Can you please try a different SDXL model and see if the issue persists?

[psychedelicious]

Ah, looks like this may be a macOS/MPS-specific issue. I've reproduced it - but with LoRAs, not IP Adapter. Hmm.

My testing is here: https://github.com/invoke-ai/InvokeAI/issues/6545

When the problem happens for you, do you get a first noisy/random preview image, then the second and all subsequent preview images are black and the final image is black?

[mjdNjhNJ]

I have the same behaviour on macOS 14.5. When using LoRas, first I get noisy image and second one plus all the following are black while also the final image is black.

[hsm207]

Just tested and con confirm the issue is gone. Thanks for the quick fix!

[psychedelicious]

@hsm207 No worries, thanks for confirming!