which has certain advantages and is often used, see e.g. here. Somewhere I found this implementation, which circumvents the computation of the cosh which easily overflows:
softplus(x::Real) = ifelse(x > 0, x + log1p(exp(-x)), log1p(exp(x)))
logcosh(x::T) where T = x + softplus(-2x) - log(convert(T, 2))
Would it be possible to include this loss function as well?
I am missing the LogCosh loss function.
which has certain advantages and is often used, see e.g. here. Somewhere I found this implementation, which circumvents the computation of the
coshwhich easily overflows:Would it be possible to include this loss function as well?