TencentARC / T2I-Adapter

T2I-Adapter
3.29k stars 196 forks source link

The training details about Unet parameters frozen #129

Open hda-xian opened 3 weeks ago

hda-xian commented 3 weeks ago

Why are the UNet parameters frozen during training for SD1v5, but not for SDXL? the haggingface training sdxl script sets " Unet.train() "

* huggingface train sdxl ***** vae.requiresgrad(False) text_encoder_one.requiresgrad(False) text_encoder_two.requiresgrad(False) t2iadapter.train() unet.train()

***Tencent ARC train sd1v5 ** model.cuda() model.eval() # model is contain all models vae ,cliptext return model

***Tencent ARC train sdxl ** vae.requiresgrad(False) text_encoder_one.requiresgrad(False) text_encoder_two.requiresgrad(False) -> the Unet does not set no grad means Unet need grad

bonlime commented 3 days ago

because they have params_to_optimize = adapter.parameters(), and only optimise the adapter's params. But disabling unet grads reduces the memory consumption, they just forgot it

hda-xian commented 3 days ago

这是来自QQ邮箱的假期自动回复邮件。你好,我最近正在休假中,无法亲自回复你的邮件。我将在假期结束后,尽快给你回复。