Closed cossio closed 2 years ago
We can enforce this constraint by parameterizing eta, for example:
eta
eta = tanh(xi)
where now xi is unbounded.
xi
We can use something faster than tanh. Here are some examples:
tanh
This suggests that the faster alternative is:
eta = xi / (1 + abs(xi))
We don't have to parameterize eta. Rather, we can use a different parameter. So, instead of using
which is required to be between -1 and 1, we can use:
which is unbounded.
The relation between the two is:
Closed in https://github.com/cossio/RestrictedBoltzmannMachines.jl/commit/e774670e59889803b483f9da25f734d78d3936c7
We can enforce this constraint by parameterizing
eta
, for example:where now
xi
is unbounded.We can use something faster than
tanh
. Here are some examples:This suggests that the faster alternative is: