Dead simple example of LSTM

[basickarl]

I've even put up a question on stackoverflow, your more than happy to answer!

https://stackoverflow.com/questions/43574799/dead-simple-example-of-synaptic-js-lstm-rnn-algorithm

As the subject states, no full examples of how to A) Train then B) Test. CLEAR examples, there are complicated half done examples, nothing I can wrap my head around though.

[basickarl]

const lstm = new Architect.LSTM(1, 4, 4, 4, 1, options);

Default options for the LSTM (should ALSO be stated in the documentation...).

peepholes: Layer.connectionType.ALL_TO_ALL
hiddenToHidden: false
outputToHidden: false
outputToGates: false
inputToOutput: true

Already has it's own trainer. No need for const trainer = new Trainer(lstm);, just call it const results = lstm.trainer.train(set, options);.

Default trainer options (should ALSO be stated in the documentation...):

this.rate = options.rate || .2;
this.iterations = options.iterations || 100000;
this.error = options.error || .005;
this.cost = options.cost || null;
this.crossValidate = options.crossValidate || null;

The source code is also not written well, what are public methods? What are private?

feed-forward activation of all the layers to produce an ouput (spelling mistake in source code):

const results = lstm.activate(input);

All input data training or testing MUST be normalized (value between 0 and 1).

[basickarl]

https://github.com/cazala/synaptic/wiki/Normalization-101 https://github.com/cazala/synaptic/wiki/Trainer

[basickarl]

Finally:

const synaptic = require('synaptic');

const Architect = synaptic.Architect;
const Layer = synaptic.Layer;

const lstmOptions = {
    peepholes: Layer.connectionType.ALL_TO_ALL,
    hiddenToHidden: false,
    outputToHidden: false,
    outputToGates: false,
    inputToOutput: true,
};
const lstm = new Architect.LSTM(1, 4, 4, 4, 1, lstmOptions);

const trainSet = [
    { input: [0], output: [0] },
    { input: [1], output: [1] },
    { input: [1], output: [0] },
    { input: [0], output: [1] },
    { input: [0], output: [0] },
];
const trainOptions = {
    rate: 0.2,
    iterations: 10000,
    error: 0.005,
    cost: null,
    crossValidate: null,
};
const trainResults = lstm.trainer.train(trainSet, trainOptions);
console.log(trainResults);

const testResults = [];
testResults[0] = lstm.activate([0]);
testResults[1] = lstm.activate([1]);
testResults[2] = lstm.activate([1]);
testResults[3] = lstm.activate([0]);
testResults[4] = lstm.activate([0]);
console.log(testResults);

Thanks to https://github.com/wagenaartje/neataptic/wiki/Visualising-101 that actually has example code which is pedagogic.

[basickarl]

const trainSet = [
    { input: [0], output: [0.1] },
    { input: [1], output: [0.2] },
    { input: [0], output: [0.3] },
    { input: [1], output: [0.4] },
    { input: [0], output: [0.5] },
];
// ...
testResults[0] = lstm.activate([0]);
testResults[1] = lstm.activate([1]);
testResults[2] = lstm.activate([0]);
testResults[3] = lstm.activate([1]);
testResults[4] = lstm.activate([0]);

Results in:

[ [ 0.18288280009908592 ],
  [ 0.2948083898027347 ],
  [ 0.35061782593064206 ],
  [ 0.3900799575806566 ],
  [ 0.49454852760556606 ] ]

[jeancode]

Este código reconoce letras en canvas pero no sirve

Control