search

algorithmiaio / danku

Exchange ML models in a trustless manner!
148 stars 30 forks source link

Confusing Logic in forward_pass2 Function of Danku.sol #9

Open doublespending opened 6 years ago

doublespending commented 6 years ago

According to the logic of forward_pass2 function of Danku.sol, total += biases[layer_i];means that I have two biases if there are only one hidden layer. In other world, the length of array biases only depends on the number of layers.

However, one biases is assigned to one neuron in your paper and package "dutils.neural_network". In other world, the length of array biases depends on the number of neuron.

For example, I trained the model with package dutils.neural_network. 

Biases: [-30, -2279, -1054, -130, -409, 152, -202, -2148, -1131, -116, -1352, 651, -1991, 532, 1628, 173, 97, -1652, 757, 1286, 1253, -437, 933, 1367, -411, -737, 1340, -867, -1320, -2536, 1, -477, -567, 521, 474, 616, 697, -1528, -11, 726, -225, -375, -379, -481, -315, 483, -1146, 1432, 692, 441, 2462, 1627, 1416, -2090, -886, 2153, -234, 1109, -54, 571, 333, -658, -3385, 759, -1271, 150, 886, -376, -202, 996, 2441, 671, -873, -1204, -721, -583, -568, -515, -574, -435, 1044, 1192, -2322, -1304, -971, 300, 2627, 138, -1225, 836, 1444, 920, 1098, 1882, 292, 1439, -868, 891, -156, 2331, -209, -1158, -419, -236, -1187, 452, -626, 329, 413, -2374, -150, -472, -23, 1281, -299, -330, -1208, 1514, 428, 178, -747, -1304, -269, 581, 725, 2716, -701, 308, 281, 713, -572, -360, -1743, -429, -638, -262, -547, -884, 1554, -50, 682, -153, -971, -2264, 404, -1769, 298, -1982, 923, 10, -27, 401, -1803, -519, -379, 316, -247, -1580, 1068, -1400, -2691, -729, -388, -1092, -177, 955, 461, -415, 1221, 1165, 84, -88, -863, 666, 1198, -1702, -932, -320, 209, -17, 998, 182, 569, -252, 1157, 517, -1669, 0, 495, -537, 336, -417, 189, 35, 1149, -716, 167, 741, -2216, 2057, -376, 959]

Num_neurons_input_layer: 2

Num_neurons_output_layer: 2

Num_neurons_hidden_layer: [100, 100]

According to the logic of foward_pass2 function, I should only get array biases with only two biases.

besirkurtulmus commented 6 years ago

@doublespending That's a good question.

Generally speaking, the way biases are implemented in neural networks is that there's a bias for every neuron in a forward passing manner.

There are no biases for input neurons, therefore you should have 100 (hidden 1) + 100 (hidden 2) + 2 (output) biases. Every forward-pass neuron has its own bias.

If you count the number of biases you have up there, it should add up to 202.

The code snippet you've provided just adds its bias to the output value of the neuron.

besirkurtulmus commented 6 years ago

I'm closing this issue since there's been no response in the last 7 days.

besirkurtulmus commented 6 years ago

I'm reopening this issue, because apparently I can't see things in plain sight.

The bias implementation stores all of the biases for each neuron, but only uses N_layer of them during a forward pass, hence it only uses 1 bias value per layer.