Moved: Shared weights

[dan-ryan]

This suggestion was moved from Neataptic. https://github.com/wagenaartje/neataptic/issues/54

In order to amplify the results of neural networks on larger datasets (i.e. MNIST), we need to implement shared weights. This will also allow the creation of convolutional networks.

[luiscarbonell]

Just putting this here for near future reference:

• https://arxiv.org/abs/1510.00149
• http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.57.4343&rep=rep1&type=pdf
• https://datascience.stackexchange.com/questions/26755/cnn-how-does-backpropagation-with-weight-sharing-work-exactly
• https://www.quora.com/What-exactly-is-meant-by-shared-weights-in-convolutional-neural-network

[raimannma]

With shared weights we could evolve convolutional neural networks. But I think we need speciation, to really evolve those.

[raimannma]

I want to use atomic numbers like the AtomicInteger class in Java.

So we can set the same atomic number to each node in a bundle with weight sharing. And all nodes could use and change the weight.

A first implementation could be found here: https://github.com/raimannma/carrot/blob/add-convolution/src/utils/Atomic.ts

We could also use this, to share biases and other things through many nodes / connections.

• [x] Implement generic atomic class
• [x] Implement number atomic class
• [ ] Use atomics for shared weights and biases