Training a neural network with continuous data from acclerometer

[SrujithPoondla]

Hi, I found synaptic very useful for building multi layer networks and few other things also. Now i am trying to train a network with the accelerometer data and want to predict the accelerometer reading in the next 50ms time. But i found that the trainer uses one hot encoding in synaptic which is not useful for this(i guess). So is there any other way to use the trainer to train with continuous data from accelerometer with out normalizing?

[wagenaartje]

It's a little unclear what your problem is. Why exactly isn't it possible to train with continuous data? Besides, training with continuous data is less effective than gathering a large amount of data and shuffling it and training it with a trainer.

[SrujithPoondla]

@wagenaartje My only problem is, i couldn't find the correct training data set format. one more question is how to use trainer to train with continuous data. Although there is one link describing how to use trainer it is not clear for continuous data training

[wagenaartje]

Oke. Just to settle a point; you cannot use the trainer for continuous data training. You can however continuously backpropagate, but that is not advised as you should shuffle your training data. So this is the way I would do it:

SET_SIZE = 100;

var network = new synaptic.Architect.Perceptron(3,9,3);
var trainer = new synaptic.Trainer(network);
var trainingSet = [];

var p_x = accel.getX();
var p_y = accel.getY();
var p_z = accel.getZ();

while(training){
  var x = accel.getX();
  var y = accel.getY();
  var z = accel.getZ();

  // Push trainingset
  trainingSet.push({input: [p_x,p_y,p_z], output: [x,y,z]});

  if(trainingSet.length == SET_SIZE){
    // Train the network with shuffle
    trainer.train(trainingSet, {
      rate: 0.1,
      error: 0.0005,
      iterations: 50,
      shuffle: true,
      log: 1,
      cost: synaptic.Trainer.cost.MSE
    });

    var ri = Math.floor(Math.random() * trainingSet.length); // Random Index position in the array
    trainingSet.splice(ri, 1); // Splice out a random element using the ri var
  }

  // Save previous values
  var p_x = x;
  var p_y = y;
  var p_z = z;
}

I never ran this code so see it as pseudocode.

What this code essentialy does is save the previous and current accelometer values, pushes them in the trainingset and waits until the trainingset reaches a certain size: then it trains the network while shuffling the data. Afterwards 1 element is removed from the set so it stays the same size.

Depending on your training environment, I'd say a SET_SIZE of ~500 is useful, especially if you measure every 50 ms.

[SrujithPoondla]

@wagenaartje thanks for your suggestion. Accelerometer data should not be shuffled because that is the trend which I wanna predict in final. One more thing i wanna mention is i will be having data for 2 ms. Can i use LSTM here ?

[wagenaartje]

You understand that shuffling the training data doesn't make your NN not predict a trend?

But I also think you don't have to train with continuous data. Please explain why you would do that. You could just make an enormous but varying training set and backpropagate that numerous of times. Learning continous data has an advantage when the environment is constantly changing though.

I'm not really familiar with network types, so I can't give any advice on network types. You could use LSTM ofcourse.

You should just experiment with different network types and see which one works out the best.

[andrew-mestas]

@SrujithPoondla Without getting into specifics what is your intended output of your network when provided acceleration values?

Are you saying you want to classify the direction of acceleration currently or in the future?

This is just side points, but given that you are measuring acceleration it can be in 3 states increasing, decreasing, or constant/rest. Since you have time series data you might be able to approach it by considering the trend of values using anomaly detection to detect chages in acceleration. Or possibly by taking the slope of a single interval vs the most recent average from slope change and compare the predicted next value to the actual value by this method you can constantly fine tune your predictions.

[SrujithPoondla]

@wagenaartje Thanks for your advice. @andrew-mestas My intended output is to continuously predict the accelerometer values 200ms ahead. I am able to achieve this in python using recurrent neural network. But i wanted to try this in javascript so I am trying to use this library.

[Jabher]

Good advices were given here, I can only give one more advice - to use quaternions to represent accelerometer data.