Hey, could you check my code for the exponential linear activation? (x if x > 0 and alpha * (exp(x)-1) if x < 0, arxiv)
In our case, elu shows much better accuracy in comparison with relu.
Probably there are ways to speed up exp operation as it has done with RoughSigmoid. What do you think?
Hey, could you check my code for the exponential linear activation? (
x if x > 0andalpha * (exp(x)-1) if x < 0, arxiv) In our case, elu shows much better accuracy in comparison with relu.Probably there are ways to speed up exp operation as it has done with RoughSigmoid. What do you think?