Training ControlNet + IP Adapter at the same time

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.

Apache License 2.0

4.48k stars 293 forks source link

Closed Yunski closed 3 months ago

Yunski commented 3 months ago

Do I need to set the attn processor when I train controlnet + ip adapter, like in the following code?

controlnet = ControlNetModel.from_pretrained(...)
controlnet.set_attn_processor(CNAttnProcessor())
controlnet.train()

xiaohu2015 commented 3 months ago

you don't need this, you can just feed text features to controlnet.

Yunski commented 3 months ago

Thanks!