search

naturomics / CapsLayer

CapsLayer: An advanced library for capsule theory
Apache License 2.0
362 stars 117 forks source link

Reproducing results from "Matrix Capsules with EM Routing" #30

Open ashleygritzman opened 5 years ago

ashleygritzman commented 5 years ago

Using this implementation, has anyone managed to get close to the results reported in the paper on the smallNORB dataset?

naturomics commented 5 years ago

I tested it on MNIST and CIFAR10, but not smallNORB since I found that matrix capsnet is worse than dynamic routing capsnet:

  1. matrix capsnet is easy to overfit. training accuracy on cifar10 converges to 100% but validation accuracy is just about 60%, which is even worse than a simple CNN model and takes more time(almost 2xepochs) to converge. matrix capsnet paper reported they got 11.9% test error on cifar10.
  2. Comparison of these models with different training size of mnist can be found here, the matrix capsnet is disappointing.
vinoo999 commented 5 years ago

Were the metrics you posted above for vectorCapsNet using your previous implementation CapsNet-Tensorflow or using vectorCapsNet? Running this code I'm only hitting about 75% accuracy running from scratch. MatrixEM is also prohibitively slow for the full dataset but I think that's more a product of the actual algorithm than the implementation. I did have to change the DataLoader configuration as for some reason loading the shuffle buffer ended up being much larger than it intuitively should've been and fully occupied my machine's RAM.