using evolve on LTSM

[tylerlindell]

is it possible to use evolve on a network such as LTSM? If so, how? otherwise, what's an alternative?

[wagenaartje]

This is possible of course. But this doesn't necessarily mean that it will be better than just starting from 'scratch'.

This is an example where a LSTM network learns to perform as a XOR gate (which isn't a recurrent data set, it just serves as an example).

const { methods, architect } = neataptic;

var network = new architect.LSTM(2, 4, 1);

const dataSet = [
  { input: [0,0], output: [0] },
  { input: [0,1], output: [1] },
  { input: [1,0], output: [1] },
  { input: [1,1], output: [0] }
];

network.evolve(dataSet, {
  log: 50,
  iterations: 500,
  mutationRate: 0.5,
  clear: true,
  equal: true,
  error: 0
});

Run here

[tylerlindell]

Reason I wanted to try it out is because my LSTM is running very slow with more than a single line of text. When I used the evolve method though without the LSTM, it was working more quickly. (Could be due to the number of layers though, not sure). My other thought was to get the LSTM running on GPU which would likely be much faster. I have also considered training on an LSTM outside of neataptic then importing the values into neataptic to be used there. maybe you have some suggestion that could help :-)

Thank you by the way for putting this together, I really like it!

[allbinmani]

@tylerlindell : I just submitted a PR to increase training speed of LSTMs significantly (tho not yet to where wikitext-103 training is plausible.. ) https://github.com/wagenaartje/neataptic/pull/89