search

znxlwm / UGATIT-pytorch

Official PyTorch implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation
MIT License
2.51k stars 476 forks source link

No activation between residual blocks #35

Open moetayuko opened 5 years ago

moetayuko commented 5 years ago

There's neither relu before the first conv https://github.com/znxlwm/UGATIT-pytorch/blob/f7a4b596a32349c54acc55007dce4ae4400bf125/networks.py#L118 nor after addition https://github.com/znxlwm/UGATIT-pytorch/blob/f7a4b596a32349c54acc55007dce4ae4400bf125/networks.py#L129 in the residual blocks, i.e., there're no activations between neighboring residual bocks.

I failed to find design details of residual blocks from the paper, but I guess it's a bug, can you confirm? Thanks.

PS: Just checked the official tf code, it suffers from the same problem.

heartInsert commented 4 years ago

I don't think it's crucial whether there exist a relu function after the first conv.

Pikauba commented 4 years ago

The addition acts like a non-linearity already. It's a design choice, you can see the same in Cycle-GAN and pix-to-pixHD. It actually allows it to output negative values.