Can I use it for handwriting recognition

[vudangthinh]

I need to use this library to recognize handwriting, but i don't know how to train data. Can you help me? Thanks!

[gianlucabertani]

Training a network for image recognition works by giving an image as input and the correct classification (e.g. the character) as expected output. That is:

• your network should have an input layer as big as the image of a character you expect: for images of 100x100 pixels you should have 10000 input neurons, where successive rows of pixels are simply juxtaposed (row 0 goes to neurons 0 to 99, row 1 goes to neurons 100 to 199, etc); consider that computation time grows geometrically with layer sizes, so keep images small;
• it should have an output layer as big as the number of characters your expect: if you want it to recognize the common upper case latin alphabet, you should have 26 output neurons (letter "A" at neuron 0, letter "B" at neuron 1, etc.); use the logistic function for output, which suits well for classification.

Once the network has been setup, the training loop should be as follows:

1. pick an image/character pair;
2. fill the input layer with the image (just light intensity, no colors);
3. compute the output;
4. fill the expectedOutput layer with the correct class (e.g. 1.0 for the neuron at the position of the expected character, 0.0 for others);
5. back propagate;
6. update weights;
7. go to 1, but at end of the training set check overall accuracy and quit if high enough.

Beware: image recognition is hard, the network may not converge (i.e. accuracy never improves) or accuracy may never get to the desired threshold (i.e. stay below 50%), depending mainly on the images. There's a lot of research on this subject, commonly used tricks to improve accuracy are:

• introduce hidden layers (bigger than input, smaller than input, one bigger and one smaller, etc.);
• preprocess the image (enhance contrast, enhance borders, etc.);
• expand the training set by submitting the same image multiple times with simple transformations (e.g. rotate by 15°, then rotated by 30° o -15°, etc.).

Best results are usually obtained with convolutional neural networks, which are a specific kind of network that my library still does not support. Anyway, I am completing the next version of the library, which includes an improved algorithm for back propagation (RPROP+) and an improved cost function for logistic neurons (cross entropy). If your network does not converge these may come to the rescue, but I can't make a prediction on when they will be released. Hopefully within a couple of weeks.

Hope this helps. Let me know your results.

[vudangthinh]

Suppose I train a network on the computer, but how I can save weights of the network after training. Because I would like to set up this network on smartphone. Thank in advance

[gianlucabertani]

The NeuralNetwork class provides both a method to save the current configuration (saveConfigurationToDictionary) and to recreate the network from a saved configuration (createNetworkFromConfigurationDictionary). Once the configuration is inside an NSDictionary, it can be easily persisted to disk with the writeToFile method and reloaded with dictionaryWithContentOfFile methods.

[vudangthinh]

How can I set error threshold (Ethreshold) of this network?

[gianlucabertani]

The library does not yet provide an automatic training loop, you have to check the error by yourself at the end of each training set (point 7 above). Currently the library uses the squared error as a cost function, so the error of a training sample can be calculated in the following way:

// Somewhere before the training loop
REAL *errorBuffer= NULL;
posix_memalign((void **) &errorBuffer, 128, sizeof(REAL) * net.outputSize);

// After training the network with a sample (i.e. between point 4 and 5 above)
REAL error= 0.0;
nnVDSP_VSUB(net.outputBuffer, 1, net.expectedOutputBuffer, 1, errorBuffer, 1, net.outputSize);
nnVDSP_SVESQ(errorBuffer, 1, & error, _outputSize);
error *= 0.5;

Normally, you should some sum up the errors for each sample, then divide by the number of samples to obtain a mean squared error. Check this mean squared error to know if you have reached the desired accuracy.

[gianlucabertani]

If you're still interested on this library, the latest release includes a full example of the MNIST dataset handwritten digits recognition.