search

taesungp / swapping-autoencoder-pytorch

Official Implementation of Swapping Autoencoder for Deep Image Manipulation (NeurIPS 2020)
Other
516 stars 84 forks source link

Generator output #22

Closed tom99763 closed 2 years ago

tom99763 commented 2 years ago

I just want to ask why there's no sigmoid or tanh in the output of the generator

taesungp commented 2 years ago

We follow the formulation of StyleGAN, which doesn't use any nonlinearity to constrain the range of the RGB output. My understanding is that the model learns to constrain the range of the output values, so the absence of sigmoid or tanh doesn't make a noticeable difference.

tom99763 commented 2 years ago

I use tanh and sigmoid and found gradient vanish issue haha

I think the issue is related to their initialization method, if I change to N(0,0.02) then the issue fixed

We follow the formulation of StyleGAN, which doesn't use any nonlinearity to constrain the range of the RGB output. My understanding is that the model learns to constrain the range of the output values, so the absence of sigmoid or tanh doesn't make a noticeable difference.