Weight initialization problem

huiyegit / T2I_CL

MIT License

45 stars 16 forks source link

Weight initialization problem #4

Closed YIRuriZhongtian closed 3 years ago

YIRuriZhongtian commented 3 years ago

hi, I noticed that the code when initializing the weights is different from AttnGAN. Can you tell me the reason for doing this? https://github.com/huiyegit/T2I_CL/blob/6f749b869ac76bc6423bc319adc8f6c7c386f17b/AttnGAN%2BCL/code/miscc/utils.py#L290-L295

huiyegit commented 3 years ago

I think the weights may have the Dictionary type to store them, then we can use different keys (.i.e ''weight'' and ''weight_bar'') to get the values(weights).

YIRuriZhongtian commented 3 years ago

Thank you for your reply. Sorry, I still don’t understand the difference between these two.

    if classname.find('Conv') != -1:
        nn.init.orthogonal_(m.weight.data, 1.0)

Can I think that they have the same effect?

huiyegit commented 3 years ago

I think so. It is the same way to initialize the weight of 'Conv' layer.

YIRuriZhongtian commented 3 years ago

ok, thank you!