Closed jchhuang closed 4 years ago
The similar effect has been observed in "Dynamic Routing between Capsules" by Hinton et al.: https://arxiv.org/pdf/1710.09829.pdf . The "squashing" function in their paper (Section 2) is used for discriminating entity existence, while in our paper (Equation 6) it is leveraged to amplify reachability.
In the paper, authors mentioned Eq(6) can help amplify the effects of the reachability, why? may your help to describe the reason