Configurable NNet layers

[mikhail]

In a similar approach to what @NMO13 is doing with #182 -- I think we could have a generic NNet file (one per library) and allow individual games to set configuration for layers. One setting would specify Conv. NNet, another would specify how many times to lower the resolution etc.

I could take a stab at proof of concept if this sounds like something you'd be interested in exploring.

[suragnair]

I definitely agree this is somewhat necessary. However, I would suggest not trying to make it super flexible and generalisable. Rather, aim for the most common case for the games in the repo: 2 or 3D board with +1,0,-1 inputs.

There would be many use cases that may require additional inputs eg some scalars in addition to the board itself. Some may not be boards at all but just a 1D array.

So it may make most sense to focus on the simplest case and make an architecture that should work for most but not all. And that can be a simple architecture without too many frills. What do you think?

[mikhail]

I'll put something together as a POC, but my limited knowledge of neural networks will prevent me from making best choices in abstraction here.

[mikhail]

This started because my "board" is significantly larger than the examples here. I have a 35x70 board. With such a large board I think that only 4 layers with a kernel size 3 is learning too slow and the neural network can get trapped in a local maximum. So I wanted to start experimenting with it but it wasn't as easy as tweaking a single number.

Since I don't know neural networks very well it's very hard to figure out what the "right" choice here is. It would be great if the framework made it easy to tweak