The prediction of 0 or 1

[antonhauff]

Hi. Please tell me. I'm new in neural networks. I created a neural network architecture for the prediction of 0 or 1. First, the neural network works well - can make a mistake more than 5 times in a row, but after 1,000 predictions, it begins to make a mistake more than 10-15 times in a row. What am I doing wrong?

`var architect = require('neataptic').architect;

var trainSet = [ {input: [0], output: [1]}, //1 {input: [1], output: [1]}, //2 {input: [1], output: [0]}, //3 {input: [0], output: [1]}, //4 {input: [1], output: [1]}, //5 {input: [1], output: [0]}, //6 {input: [0], output: [0]}, //7 {input: [0], output: [1]}, //8 {input: [1], output: [1]}, //9 {input: [1], output: [1]} //10 ];

var options = { memoryToMemory: false, // default is false outputToMemory: false, // default is false outputToGates: false, // default is false inputToOutput: true, // default is true inputToDeep: true // default is true };

var LSTM = new architect.LSTM(1, 10, 1); trainSet.shift(); trainSet.push({input: [], output: []}); //recording the new data LSTM.train(trainSet, { log: 1000, iterations: 5000, error: 0.00001, rate: 0.01, clear: true });

for(var i in trainSet) { var input = trainSet[i].input; var output = Math.round(LSTM.activate([input])); }

for(var i = 0; i < 10; i++){ var input = output; var output = LSTM.activate([input]); }`

[kotopuz]

did you solved the problem ? I am also trying to make such prediction to work and I wonder how you managed to tune that(I see you use 10 neurons did you try to use more like 50 ?).

[ghost]

First 0 becomes 1 and then 0 becomes 0 without that you tell the NN why that is supposed to happen. Check your MSE, I assume its quite high. Your training data needs to contain some "information" which can be learned and not some random changes. Look at the XOR example, that one contains information which the NN can learn and reproduce. Also a 10 hidden layers are not required, 2 are enuff for any problem which an LSTM can learn.

[kotopuz]

Many thanks for reply

Ive tried to use 2 hidden layers(new neataptic.Architect.LSTM(1,2,1);) and NN works really fast, but it does not predict and error is very high. IN XOR example https://jsfiddle.net/9t2787k5/3/ there are layers used and even one message say that it seems to be working fine with at least 6 layers (new neataptic.Architect.LSTM(1,6,1);)

My time series has sequnces in seqnece and sequences can be 010101 or 0001 0001 etc. I've tried to repeat them 5- 10 times, but NN does not predict well

For time series I've also found that for time series the input output structure should be: X0 X1,X2,X3 ... Xn-1, Xn

var trainingData = [ { input: [X0], output: [X1] }, { input: [X1], output: [X2] }, { input: [X2], output: [X3] }, {...}, { input: [Xn-1], output: [Xn] }, ];

Is it correct ? In some other post https://github.com/wagenaartje/neataptic/issues/97

I've seen post was: {input: [roll_prophecy], output: [roll_realy]}); var trainingData = [ { input: [X0 predicted], output: [X0}, { input: [X1 predicted], output: [X1}, { input: [X2 predicted], output: [X2] }, {...}, { input: [Xn predicted], output: [Xn] }, ];

Can you advise please ?

I've also tried to increase iterations to 25000, but still can get NN working. Please help My code below:

function PREDICT(){ var network = new neataptic.architect.LSTM(1,2,1); var trainingData = []; var arr_input = []; // sequnec is x,X+1 should be. x> input x and X+1 to output x+1 and x+1 to input x+1 then predict output x+1; it means output y = input y+1 var arr_output = []; arr_input.push(prompt("Enter a input number" + " " + i)); for(var i = 0; i < 5; i++){ arr_output.push(prompt("Enter an output number" + " " + i)); arr_input.push(arr_output[arr_output.length - 1]); // next input is last output value in array } arr_output.push(prompt("Enter last output number")); console.log("input array: " + arr_input.join(', '));// alert the results console.log("output array: " + arr_output.join(', '));// alert the results

// create initial training data
for(var iii in arr_input){
 trainingData.push({
 input: [arr_input[iii]],
 output: [arr_output[iii]]
 });
 }

// JAVASCRIPT loop MYLOOP: do {

arr_input.push(arr_output[arr_output.length - 1]); // this ouput (last

array element) should be equal to next input arr_output.push(prompt("Emulate next time series value new value 1 or 0")); // next time series value emulation from user input console.log("new time series: " + arr_output[arr_output.length - 1]);

trainingData.push({
 input: [arr_input[arr_input.length - 1]],
 output: [arr_output[arr_output.length - 1]]
 });

network.train(trainingData, {
  log: 5000,
  iterations: 25000,
  error: 0.0003,
  clear: true,
  rate: 0.05,
});
//
for(var iiii in trainingData){
  var input = trainingData[iiii].input;
  var output = Math.round(network.activate([input]));
  console.log("Num: " + iiii + "  Input: " + input + ", output: " +

output + "!!!"); } console.log("training data lengh: " + trainingData.length ); input = output; console.log("Next prediction: " + Math.round(network.activate([input]))

• "!!!");

  //if (prompt("1 - exit, 0 -continue")==1) break MYLOOP; continue MYLOOP;// Not necessary since you can just put do {} while (1) but it illustrates } while (1) };

On Sat, Dec 22, 2018 at 12:09 AM Pummelchen notifications@github.com wrote:

First 0 becomes 1 and then 0 becomes 0 without that you tell the NN why that is supposed to happen. Check your MSE, I assume its quite high. Your training data needs to contain some "information" which can be learned and not some random changes. Look at the XOR example, that one contains information which the NN can learn and reproduce. Also a 10 hidden layers are not required, 2 are enuff for any problem which an LSTM can learn.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wagenaartje/neataptic/issues/106#issuecomment-449520342, or mute the thread https://github.com/notifications/unsubscribe-auth/AhGYCJAMiHG0VfPLpsosecurKr-P-qdWks5u7WongaJpZM4Qr3M5 .

[kotopuz]

Many thanks for reply

Ive tried to use 2 hidden layers(new neataptic.Architect.LSTM(1,2,1);) and NN works really fast, but it does not predict and error is very high. IN XOR example https://jsfiddle.net/9t2787k5/3/ there are layers used and even one message say that it seems to be working fine with at least 6 layers (new neataptic.Architect.LSTM(1,6,1);)

My time series has sequnces in seqnece and sequences can be 010101 or 0001 0001 etc. I've tried to repeat them 5- 10 times, but NN does not predict well

For time series I've also found that for time series the input output structure should be: X0 X1,X2,X3 ... Xn-1, Xn

var trainingData = [ { input: [X0], output: [X1] }, { input: [X1], output: [X2] }, { input: [X2], output: [X3] }, {...}, { input: [Xn-1], output: [Xn] }, ];

Is it correct ? In some other post https://github.com/wagenaartje/neataptic/issues/97

I've seen post was: {input: [roll_prophecy], output: [roll_realy]}); var trainingData = [ { input: [X0 predicted], output: [X0}, { input: [X1 predicted], output: [X1}, { input: [X2 predicted], output: [X2] }, {...}, { input: [Xn predicted], output: [Xn] }, ];

Can you advise please ?

I've also tried to increase iterations to 25000, but still can get NN working. Please help My code below:

function PREDICT(){
    var network = new neataptic.architect.LSTM(1,2,1);
    var trainingData = [];
    var arr_input = [];  // sequnec is x,X+1 should be. x> input x and X+1 to output x+1 and x+1 to input x+1 then predict output x+1; it means output y = input y+1
    var arr_output = [];
    arr_input.push(prompt("Enter a input number" + " " + i));
    for(var i = 0; i < 5; i++){
     arr_output.push(prompt("Enter an output number" + " " + i));
     arr_input.push(arr_output[arr_output.length - 1]); // next input is last output value in array
    }
    arr_output.push(prompt("Enter last output number"));
    console.log("input array: " + arr_input.join(', '));// alert the results
    console.log("output array: " + arr_output.join(', '));// alert the results

    // create initial training data
    for(var iii in arr_input){
     trainingData.push({
     input: [arr_input[iii]],
     output: [arr_output[iii]]
     });
     }
// JAVASCRIPT loop
MYLOOP: do {

    arr_input.push(arr_output[arr_output.length - 1]); // this ouput (last array element) should be equal to next input
    arr_output.push(prompt("Emulate next time series value new value 1 or 0")); // next time series value  emulation from user input
    console.log("new time series:   " + arr_output[arr_output.length - 1]);

    trainingData.push({
     input: [arr_input[arr_input.length - 1]],
     output: [arr_output[arr_output.length - 1]]
     });

    network.train(trainingData, {
      log: 5000,
      iterations: 25000,
      error: 0.0003,
      clear: true,
      rate: 0.05,
    });
    //
    for(var iiii in trainingData){
      var input = trainingData[iiii].input;
      var output = Math.round(network.activate([input]));
      console.log("Num: " + iiii + "  Input: " + input + ", output: " + output + "!!!");
    }
    console.log("training data lengh: " + trainingData.length );
    input = output;
    console.log("Next prediction: " + Math.round(network.activate([input])) + "!!!");

  //if (prompt("1 - exit, 0 -continue")==1) break MYLOOP;
  continue MYLOOP;// Not necessary since you can just put do {} while (1) but it     illustrates
} while (1)
};