search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.2k stars 5.21k forks source link

IP adapter plus not working #6131

Closed alexblattner closed 8 months ago

alexblattner commented 9 months ago

Describe the bug

clip_embeddings_dim = state_dict["image_proj"]["proj.weight"].shape[-1]
KeyError: 'proj.weight'

Reproduction

image1 = Image.open('a.jpg').convert('RGB')

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
)
pipe.to(torch_device="cuda", torch_dtype=torch.float32)

pipe.load_ip_adapter("h94/IP-Adapter", subfolder="models", weight_name="ip-adapter-plus_sd15.bin",cache_dir="model_cache")

generator = torch.Generator(device="cpu").manual_seed(3635017627)
images=pipe(prompt="mona lisa",width=512,height=512,generator=generator,num_inference_steps=28,guidance_scale=7,ip_adapter_image=image1).images

Logs

No response

System Info

default one

Who can help?

@yiyixuxu @DN6 @sayakpaul @patrickvonplaten

alexblattner commented 9 months ago

this is the latest main version btw

zhaoyun0071 commented 9 months ago

+1 same problem

DN6 commented 9 months ago

@yiyixuxu Could you take a look please?

yiyixuxu commented 9 months ago

sorry I can't reproduce this issue

alexblattner commented 9 months ago

@yiyixuxu is there anything I could provide to help? I have the latest version of diffusers and it doens't work...

zhaoyun0071 commented 9 months ago

@yiyixuxu is there anything I could provide to help? I have the latest version of diffusers and it doens't work...

IP Adapter Plus and IP Adapter Full are not supported in diffusers 0.24.0. If you want to use one of them, you can clone the repository and use the current version

github-actions[bot] commented 8 months ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.