huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.03k stars 5.17k forks source link

The training details about Unet parameters frozen (and no script for train sd1.x) #8451

Open hda-xian opened 2 months ago

hda-xian commented 2 months ago

Describe the bug

Why are the UNet parameters frozen during training for SD1v5, but not for SDXL? the haggingface training sdxl script sets " Unet.train() "

Reproduction

diffusers : 0.28 * huggingface train sdxl ***** vae.requiresgrad(False) text_encoder_one.requiresgrad(False) text_encoder_two.requiresgrad(False) t2iadapter.train() unet.train()

***Tencent ARC train sd1v5 ** model.cuda() model.eval() # model is contain all models vae ,cliptext return model

***Tencent ARC train sdxl ** vae.requiresgrad(False) text_encoder_one.requiresgrad(False) text_encoder_two.requiresgrad(False) -> the Unet does not set no grad means Unet need grad

Logs

No response

System Info

Ubuntu

Who can help?

No response

sayakpaul commented 2 months ago

Can you point me to the line of code you're referring to?

hda-xian commented 2 months ago

这是来自QQ邮箱的假期自动回复邮件。你好,我最近正在休假中,无法亲自回复你的邮件。我将在假期结束后,尽快给你回复。