search

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.
Apache License 2.0
4.53k stars 298 forks source link

using pipe.load_lora_weights breaks model loading #141

Open bitcoinvsalts opened 8 months ago

bitcoinvsalts commented 8 months ago

If I use load_lora_weights it breaks the ip adapter model loading.

Code: pipe.load_lora_weights("latent-consistency/lcm-lora-sdxl") ... ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, accelerator.device)

Error log: AttributeError: 'UNet2DConditionModel' object has no attribute 'attn_processors'

Any idea why?

is IP Adapter compatible with diffusers v.0.23?

xiaohu2015 commented 8 months ago

I will check that in next weekdays

Norod commented 7 months ago

I also got this error when I tried with the latest diffusers and transformers versions. Rolling back to: diffusers-0.21.1 tokenizers-0.13.3 transformers-4.33.2 Allowed me to load my Kohya SDXL LoRAs to the pipeline and use them with IPAdapter for SDXL

Daming-TF commented 6 months ago

I also found the same problem, I trace the problem is finally found UNet2DConditionModel attn_processors parameters of a problem, Just change "module.get_processor(return_deprecated_lora=True)" to "module.get_processor(return_deprecated_lora=False)" it worked