search

naturomics / CapsNet-Tensorflow

A Tensorflow implementation of CapsNet(Capsules Net) in paper Dynamic Routing Between Capsules
Apache License 2.0
3.8k stars 1.17k forks source link

About the routing biases #77

Closed ChristofferM closed 5 years ago

ChristofferM commented 5 years ago

In 5b4eb1a biases for s_j were added. How is this motivated? It is not in the NIPS-paper? Also shouldn't B_IJ be learned?

naturomics commented 5 years ago

The paper didn't mention this. The biases term is motivated by the fact that capsule is essentially an extend of neuron from two aspect: 1. the separation of coupling assignment c from weight w; 2. matrix representation of x_i instead of scalar

$y_j=f\big(\sum_i(c_{ij}w_{ij})x_i+b_j\big)=f(\sum_i \tilde{w}_{ij}x_i+b_j)$

the coupling assignment c is learned by routing algorithm, and w by back-propagation. It's not mentioned in paper, but similar to neuron, a bias term for intercept would be more reasonable. For a more intuitive comparison of capsule and neuron, see the picture in readme. We will soon publish our work discussing the relation of capsule and neuron in more detail, welcome to follow

ChristofferM commented 5 years ago

Thanks! Looking forward to it.