Closed cossio closed 2 years ago
If the unit has zero activation, the dReLU energy should give zero.
dReLU
Currently, since the dReLU energy is implemented by calling the ReLU energy, it gives Inf.
ReLU
Inf
This should be corrected.
Edit: Actually this works fine, because the ReLU energy at zero is zero. The ReLU energy gives Inf only if x < 0.
x < 0
If the unit has zero activation, the
dReLUenergy should give zero.Currently, since the
dReLUenergy is implemented by calling theReLUenergy, it givesInf.This should be corrected.
Edit: Actually this works fine, because the ReLU energy at zero is zero. The ReLU energy gives
Infonly ifx < 0.