Layer Quantity and Quality

[stefanpeidli]

How many Layers should the NN have? And how large should they be?

Up to now, the only known method is experimenting...

[paddyooo]

Passen als 82ter Eintrag in TrainingData.dic.Value und als 82ter output im NN

[stefanpeidli]

In the original AlphaGo Natur Paper (Page 8) we see:

Neural network architecture. The input to the policy network is a 19×19×48 image stack consisting of 48 feature planes. The first hidden layer zero pads the input into a 23×23 image, then convolves k filters of kernel size 5×5 with stride 1 with the input image and applies a rectifier nonlinearity. Each of the subsequent hidden layers 2 to 12 zero pads the respective previous hidden layer into a 21×21 image, then convolves k filters of kernel size 3×3 with stride 1, again followed by a rectifier nonlinearity. The final layer convolves 1 filter of kernel size 1×1 with stride 1, with a different bias for each position, and applies a softmax func- tion. The match version of AlphaGo used k=192 filters; Fig. 2b and Extended Data Table 3 additionally show the results of training with k =128, 256 and 384 filters. The input to the value network is also a 19×19×48 image stack, with an addi- tional binary feature plane describing the current colour to play. Hidden layers 2 to 11 are identical to the policy network, hidden layer 12 is an additional convolution layer, hidden layer 13 convolves 1 filter of kernel size 1×1 with stride 1, and hidden layer 14 is a fully connected linear layer with 256 rectifier units. The output layer is a fully connected linear layer with a single tanh unit.

Thus the input size of the Policy Network is a stunning 19×19×48=17328, followed by around 13 hidden layers.

[stefanpeidli]

To Do: Write a Script for testing of performance of a Net on a training set. Use this then to choose a best arquitecture. See 4.2 Training in Heining.