cazala / synaptic

architecture-free neural network library for node.js and the browser
http://caza.la/synaptic
Other
6.92k stars 666 forks source link

Impossible train Pacal triangle this way #199

Open iEfimoff opened 7 years ago

iEfimoff commented 7 years ago

https://en.wikipedia.org/wiki/Pascal's_triangle

I've never used neural network before.

var myNetwork = new Architect.Perceptron(20, 20, 10)
var trainer = new Trainer(myNetwork)

var trainingSet = [
  {
    input: [1],
    output: [1]
  },
  {
    input: [2],
    output: [1,1,1]
  },
  {
    input: [3],
    output: [1,1,1,1,1,2,1]
  },
  {
    input: [4],
    output: [1,1,1,1,1,2,1,1,3,3,1]
  },
  {
    input: [5],
    output: [1,1,1,1,1,2,1,1,3,3,1,1,4,6,4,1]
  }
]

trainer.train(trainingSet,{
    rate: .1,
    iterations: 20000,
    error: .005,
    shuffle: true,
    log: 1000,
    cost: Trainer.cost.CROSS_ENTROPY
});
venil7 commented 7 years ago

if you're defining perceptron with 20 in, and 10 out neurons, your training dataset should match

    // input.length === 20 [0..1]
    // output.length === 10 [0..1]
wagenaartje commented 7 years ago

You are trying to train it in the wrong way. A neural network is not going to input a whole triangle as you want. You could however teach the neural network the formula:

formula I don't know if it's the correct one but just study the wikipedia page

If you really want it to output the whole triangle, you could make something as follows (which just memorizes the layers), this is for a 6 layer triangle:

var network = new synaptic.Architect.Perceptron(1, 3, 6);
var trainer = new synaptic.Trainer(network);

// input: layer x divided by total amount of layers
// output: all the numbers of the layer in order, however not contered (there is no middle of 6)
// divide output by 10 because output should be between 0 and 1
var trainingSet = [
  {
    input: [1 / 6],
    output: [1/10,0,0,0,0,0] 
  },
  {
    input: [2 / 6], 
    output: [1/10,1/10,0,0,0,0]
  },
  {
    input: [3 / 6],
    output: [1/10,2/10,1/10,0,0,0]
  },
  {
    input: [4 / 6], 
    output: [1/10,3/10,3/10,1/10,0,0]
  },
   {
    input: [5 / 6], 
    output: [1/10,4/10,6/10,4/10,1/10,0]
  },
  {
    input: [6 / 6], 
    output: [1/10,5/10,10/10,10/10,5/10,1/10]
  },
];

trainer.train(trainingSet,{
    rate: .1,
    iterations: 20000,
    error: .0005,
    shuffle: true,
    cost: synaptic.Trainer.cost.MSE
});

for(var i = 0; i < 6; i++){
  var result = network.activate([(i+1)/6]);
  for(var j = 0; j < result.length; j++){
    result[j] = Math.round(result[j] * 10);
  }
  console.log('Layer ', i+1, ' results ', result);
}

Note that the above code is not very useful. I advise you to try to teach the neural network to actually memorize the formula so you could input the layer n and receive that layers numbers.

Jabher commented 7 years ago

Thing is you're trying to build model, which is answering "what will this triangle look like", however, it will not work in that way. This triangle is, actually (height: number) => OrderedGraph<number>, and inside this fn - there is something like (height, top = triangle(height - 1)) => height <= 1 ? [1] : [...top, computeLayer(height, top)]. Also, layer internals (in computeLayer) is also function of height and index within this layer. So, you probably would like to predict the value inside some specific cell, and fill array with this. This means your model should predict the following function: (layer: number, positionInLayer: number) => number. If you decompose the NN purpose in that way, keeping in mind that NN is, actually "scalar | fixed array | fixed n-dimensional array in, scalar | fixed array | fixed n-dimensional array out" type, everything will get extremely simple.