search

DariusAf / MesoNet

"MesoNet: a Compact Facial Video Forgery Detection Network" (D. Afchar, V. Nozick) - IEEE WIFS 2018
Apache License 2.0
251 stars 112 forks source link

About the leakyReLU #25

Closed CauchyComplete closed 4 years ago

CauchyComplete commented 4 years ago

Hi, it seems that your implementation has a leaky ReLU between two fully connected layers in both Meso4 and MesoInception4. But figure 4 in your paper does not have a leaky ReLU. Why the difference? Thanks

DariusAf commented 4 years ago

Hi, Yes you are right. The training of the model dates back two years ago so I cannot say for sure, but I suppose this was an implementation detail I did not mention in the paper because of a lack of space. If I remember, the models were a bit less performant with ReLU, it was a way to gain one or two points of accuracy. As for the general differences between ReLU and LeakyReLU, you may know it's a way to avoid dead neurons for a training with high-variance gradients.

CauchyComplete commented 4 years ago

Thank you :D