search

atong01 / conditional-flow-matching

TorchCFM: a Conditional Flow Matching library
https://arxiv.org/abs/2302.00482
MIT License
1.27k stars 103 forks source link

zero_module in UNet output #134

Closed chihchiehchen closed 2 months ago

chihchiehchen commented 2 months ago

Hello,

Thanks for sharing your great work. I have a question on the self.out layer of UNet module. I found the last layer is zero_module(conv_nd(dims, input_ch, out_channels, 3, padding=1)), does it mean all the network outputs will be zero since the weights and biases of last layer of UNet are all zeros?

I know that must not be true but I do not know how to justify it. Could you give me some hints? In any case thanks for your patient and help!

atong01 commented 2 months ago

Hi, yes, this means that the network outputs will all be zero at initialization. After training, the network weights will no longer be all zeros.

chihchiehchen commented 2 months ago

I see, thanks for your help!