search

jnhwkim / cbp

Multimodal Compact Bilinear Pooling for Torch7
Other
68 stars 23 forks source link

gradient explosion in SignedSquareRoot #1

Closed jnhwkim closed 8 years ago

jnhwkim commented 8 years ago

The gradient of Signed Square Root is 1 / (2 * sqrt(abs(x))). When x is close to zero, the gradient is explosive.

Refer to https://github.com/gy20073/compact_bilinear_pooling/blob/master/MatConvNet/layers/signed_sqrt/yang_sqrt_backward.m .