cazala / synaptic

architecture-free neural network library for node.js and the browser
http://caza.la/synaptic
Other
6.91k stars 666 forks source link

Standalone function doesn't produce correct output for recurrent networks #239

Open wagenaartje opened 7 years ago

wagenaartje commented 7 years ago

The standalone function doesn't seem to work for LSTM's. The output of the standalone function is completely static. I think this is because the function actually has an array with values which it seems to reset.

See for yourself (open console)

VM93:51 [0.4669849658459853]
VM93:52 [0.466899049432868]
VM93:53 [0.4668295913678973]
VM93:54 [0.4667737478867866]
VM93:55 [0.4667290659296783]
VM93:56 [0.4666934658894935]
VM93:60 ABOVE: synaptic BELOW: standalone
VM281:62 [0.4666652089312539]
VM281:63 [0.4666652089312539]
VM281:64 [0.4666652089312539]
VM168:65 [0.4666652089312539]
VM168:66 [0.4666652089312539]
VM168:67 [0.4666652089312539]
SalvaMG commented 7 years ago

Hello ! I have de same problem with LSTM :( Please help !

SalvaMG commented 7 years ago

I tried with this simple example and it didn't work

var myNetwork = new synaptic.Architect.LSTM(2,1,1)
var trainer = new synaptic.Trainer(myNetwork);
trainer.XOR(); 
var standalone = myNetwork.standalone();

console.log(myNetwork.activate([1,0]));     
console.log(standalone([1,0]));

Result : [ 0.9127302571342819 ] [ 0.05599734668225011 ]

However when i tried after adding a new layer (var myNetwork = new synaptic.Architect.LSTM(2,1,1,1)) it worked. I think there is a problem with layer counting. There is also a comment in the source code, in standalone's definition questioning layer's count.

wagenaartje commented 7 years ago

Hmmm. Then it's a double bug. So there's 2 problems: it doens't work for recurrent networks and something is going wrong with layer counting. I shouldn't be saying this, but this library also supports LSTM networks with standalone() and it should be working fine.