How to fine-tune IP-Adapter model with xformer?

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.

Apache License 2.0

5.08k stars 331 forks source link

How to fine-tune IP-Adapter model with xformer? #329

Closed seungyonglee0802 closed 3 months ago

seungyonglee0802 commented 5 months ago

I got error message like below when I ran unet.enable_xformers_memory_efficient_attention()

You are removing possibly trained weights of IPAttnProcessor2_0 with <diffusers.models.attention_processor.XFormersAttnProcessor object at 0x7fd027803940>

Thanks in advance :)

LeonJoe13 commented 4 months ago

I got the same problem, did you fix? many thanks

JWargrave commented 4 months ago

I have found the answer

MontaEllis commented 3 months ago

I have found the answer

seems would't work

MontaEllis commented 3 months ago

The core of the error is that you called unet.enable_xformers_memory_efficient_attention() after unet.set_attn_processor(xxx). Simply switch the order.