search

cazala / synaptic

architecture-free neural network library for node.js and the browser
http://caza.la/synaptic
Other
6.92k stars 666 forks source link

Understanding clear() for LSTM #309

Closed deermichel closed 5 years ago

deermichel commented 6 years ago

Hey, can someone help me get the concept of clear()? I have the following code:

const lstm = new synaptic.Architect.LSTM(1, 1, 1)
const lstm2 = lstm.clone()

for (let i = 0; i < 1000; i++)
    lstm.activate([ 1 ])
lstm.clear()
console.log(lstm.activate([ 1 ]))

console.log(lstm2.activate([ 1 ]))

I expect the outputs to be equal since i reset the internal state of lstm after modifying it via activations. However the outputs do differ slightly... does clear() use some sort of randomized values or whats the reason behind it? Is there possibility to reset the internal state in a way so that the outputs in the upper example would be equal?

Thanks, deermichel