search

bwibbwz / ml-2048

0 stars 0 forks source link

Activation functions for each layer #7

Open bwibbwz opened 6 years ago

bwibbwz commented 6 years ago

It just came to my mind that the Sigmoid function (that I had included as the activation function for all the hidden layers) probably deals poorly with exclusively positive data (from the input layer) since it is antisymmetric around 0 and transforms values [-inf,+inf] to [0,1].

I think we'll have to dig a bit to discover the best activation functions for each layer.

bwibbwz commented 6 years ago

Adding TanH scales things a bit better (1ebed771b9). We still need to put thought into the whole setup though.