search

lmjohns3 / theanets

Neural network toolkit for Python
http://theanets.rtfd.org
MIT License
328 stars 73 forks source link

wrong NN XOR predictions #67

Closed pannous closed 9 years ago

pannous commented 9 years ago

result of first run: I 2015-02-16 21:51:11 root:21 NN XOR predictions: [[ 0. ] [ 0.5 ] [ 0.99000001] [ 0.5 ]]

oddly it worked the second time: I 2015-02-16 21:53:49 root:21 NN XOR predictions: [[ 0.] [ 1.] [ 1.] [ 0.]]

lmjohns3 commented 9 years ago

Yes, the network for this problem depends heavily on the initialization and training process for proper convergence. I'm not sure this is a bug that can be fixed with certainty, perhaps apart from seeding the random number generator.

pannous commented 9 years ago

Ah I see, thanks! Had the same issue in caffe: https://github.com/pannous/caffe/tree/dev/examples/xor_hello_world fixed it by increasing the middle layer: layers=(2, 2, 1) => layers=(2, 20, 1) so that it contains positive+negative weights with a very high likelihood

pannous commented 9 years ago

PS: fantastic framework so far, all examples work out of the box, no compilation issues etc !!