search

yulunzhang / RCAN

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"
1.37k stars 311 forks source link

one question about the activation function #8

Closed yyknight closed 5 years ago

yyknight commented 6 years ago

Thank you for your impressive work and sharing code so soon, I have read your paper RDN and RCAN ,I'm little confused about your choice of activation function, why did you always choose RELU rather than RRELU or PRELU as activation function? RELU may have many dead neurons, right?

yulunzhang commented 6 years ago

Hi, The reasons are two fold: 1. In our previous investigations (other super-resolution networks, e.g., SRResNet) about the choice of activation function, we didn't observe obvious improvement by using other activation functions, e.g., PReLU. 2.We use ReLU to keep fairer comparisions with other SR networks, e.g., EDSR. We expect the improvement comes more from network structure design than from the change of activation function.

yyknight commented 6 years ago

I see , Thanks a lot.