search

cazala / synaptic

architecture-free neural network library for node.js and the browser
http://caza.la/synaptic
Other
6.92k stars 666 forks source link

Multiple neurons gating the same connection #214

Open wagenaartje opened 7 years ago

wagenaartje commented 7 years ago

I'm not really familiar with LSTM's. However I thought I saw something odd:

//  update gated connection's gains
for (var connection in this.connections.gated) {
  this.connections.gated[connection].gain = this.activation;
}

See neuron.js (line 104)

So basically, when a neuron is activated, it will set all it's gated connections gains to its activation value - seems ok.

But what if 2 neurons gate the same connection? After an activation, the gain of a connection will bet set to one of these two neurons (depending on activation order). Does this mean that multiple neurons gating the same connection us useless?

cazala commented 7 years ago

Yes, the latter activation will be used as the gain