search

www0wwwjs1 / Matrix-Capsules-EM-Tensorflow

A Tensorflow implementation of CapsNet based on paper Matrix Capsules with EM Routing
Apache License 2.0
218 stars 83 forks source link

Back propagation withing em_routing #15

Open dipanjan06 opened 6 years ago

dipanjan06 commented 6 years ago

Hello, Thanks for releasing great code base on Capsule Network. One quick query are not stopping gradient during em_routing?

www0wwwjs1 commented 6 years ago

We have tried stopping gradient during routing. It leads to failures during training.

dipanjan06 commented 6 years ago

Ok thank you for your quick response. So my understanding is that , latent variable "r" does update during back propagation , then at E step whatever value being updated for r will overwrite during back propagation. You said that you have tried to stopping the gradient . please let know which variable are you trying stop gradient . is It votes variable?

www0wwwjs1 commented 6 years ago

Votes variables are used during different iterations of routing. It means parameters in transformation matrix should be trained to adapt different iterations. We have tried to stop gradient in the 2nd and 3rd iterations, which means parameters in transformation matrix are only trained for the initialization of routing. However, it leads to failures in training. The latent variables of EM algorithm update during forward clustering, and thus have no relationship with training.