Why output size of 100 for mnist notebook

RAIVNLab / supsup

Code for "Supermasks in Superposition"

117 stars 19 forks source link

Why output size of 100 for mnist notebook #5

Closed jamesrichter closed 4 years ago

jamesrichter commented 4 years ago

I'm going through the MNIST notebook. The output layer size for the mnist notebook looks like it's 100 and I'm wondering if there's a reason for this, seems like it should be 10.

mitchellnw commented 4 years ago

Yes, if you change it to 10 you'll find it will not work :)

Check out sec 3.6 and G in the paper for more explanation about these "superfluous neurons".