code problem

[ZhangLeiiiii]

hi, why does unet use "unet.set_adapter" method ,while vae does not?

[GaParmar]

Ah, this is because for the unet, we set three different lora adapters. This requires using the set_adapter method explicitly.

Consider this code block below for example:

from diffusers import AutoencoderKL, UNet2DConditionModel
from peft import LoraConfig

unet = UNet2DConditionModel.from_pretrained("stabilityai/sd-turbo", subfolder="unet")
lora_conf= LoraConfig(r=4, init_lora_weights="gaussian",target_modules=["to_k", "to_q", "to_v"])
unet.add_adapter(lora_conf, adapter_name="adapter_a")
unet.add_adapter(lora_conf, adapter_name="adapter_b")
unet.add_adapter(lora_conf, adapter_name="adapter_c")
print(unet.active_adapters())

This will show that only last added "adapter_c" is active.

However, to enable all three adapters, you need to set the adapters explicitly. Eg. in the code block below will show that all adapters are active.

from diffusers import AutoencoderKL, UNet2DConditionModel
from peft import LoraConfig

unet = UNet2DConditionModel.from_pretrained("stabilityai/sd-turbo", subfolder="unet")
lora_conf= LoraConfig(r=4, init_lora_weights="gaussian",target_modules=["to_k", "to_q", "to_v"])
unet.add_adapter(lora_conf, adapter_name="adapter_a")
unet.add_adapter(lora_conf, adapter_name="adapter_b")
unet.add_adapter(lora_conf, adapter_name="adapter_c")
unet.set_adapters(["adapter_a", "adapter_b", "adapter_c"])
print(unet.active_adapters())

We do not need to do this for the VAE network because it only has one adapter.

[ZhangLeiiiii]

thanks for your reply