Added Smooth Rectified Linear Unit Activation Function ln(1+exp(x)), #27

[Ithanil]

27

• New differentiable version of a Linear Rectifier Unit ( ln ( 1 + exp(x)) )
• sadly the logarithm from standard library is painfully slow... in general we should consider using a different math library (Intel MKL?)

EDIT: Following the advice from Codacy, to use log1p(x) over log(1+x), suprisingly the performance is much more acceptable.

[coveralls]

Pull Request Test Coverage Report for Build 218

• 19 of 23 (82.61%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.01%) to 83.067%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/actf/SRLUActivationFunction.hpp	1	5	20.0%
<!--	Total:	19	23	82.61%	-->

Totals
Change from base Build 200:	-0.01%
Covered Lines:	1663
Relevant Lines:	2002

---

💛 - Coveralls