Is the backbone Denoising Unet need training？

guoqincode / Open-AnimateAnyone

Unofficial Implementation of Animate Anyone

2.84k stars 228 forks source link

Is the backbone Denoising Unet need training？ #106

Open TZYSJTU opened 5 months ago

TZYSJTU commented 5 months ago

Is the backbone Denoising Unet need to be finetuned? Or it is frozen as the SD 1.5 model? In the paper, it says, "During this stage (second), we only train the temporal layer while fixing the weights of the rest of the network.‘’ But I see you said "Freezing unet is not a good choice" ? So I got confused, please answer this question, thanks！

guoqincode commented 5 months ago

Stage1 requires training and Stage2 is frozen

XingML commented 2 months ago

Stage1 requires training and Stage2 is frozen

But I find that the unet is frozen in the stage 1 in your code. Is that normal? https://github.com/guoqincode/Open-AnimateAnyone/blob/02d45006ff81b30093c9ede9f4097fe36074fa42/train_hack.py#L259