Expected outcome of project

To prepare an Artificial Neural Network wrapper on this code for string classification (mainly useful for Computational linguitstics).
To prepare a log of wrongly identified classification and train further for their proper identificaion.

Code modifications

The base code is this.

Brief overview of major modifications made by this wrapper are provided below.

Downloaded the data from http://sanskrit.uohyd.ernet.in/Corpus/SHMT/Samaas-Tagging/ in file input.txt (removed metadata).
Ran samAsa_array.php on this input.txt file to get three lists inputwords, outputwords and majorwords. (These lists are to be used by subsequent python programmes).
Copied the data of these three lists in preparation.py
In network2.py import heapq
In network2.py added functions unsolved, back_to_string, class_renamed, class_numbered and returnerror.
In network2.py SGD function, there are lines added to append unsolved_examples and they are stored in wrongdata.txt file for future manipulation.

Methodology for samAsa identification

The project started with the problem of samAsa identification for Sanskrit compounds. Its implementation details are given below.

According to this paper, there are 5 major types and 55 minor types of samAsas.
We have scrapped the samAsa tagged data available online with samAsa_array.php to get the data which we can use with the python programs.
The main code framework is the very neat and concise neural net code available here (Text and explanation of the code are here)
The code there was primarily written to identify MNIST digits classification. Input neuron had list with 784 entries ranging from 0-1. Output layer had 10 neurons corresponding to the number of digits.
The problem is almost similar in case of samAsa classification. We have arbitrarily taken 50 as maximum number of letter in a samAsa. Therefore, the input layer has 50 neurons, each corresponding to a single letter. Output layer may have 5 or 55 neurons depending on whether we want to classify only major classes of samAsa or minor ones.
Input data is was devanagari with tagging in Roman transliteration. Therefore, the letters were converted to SLP1 format for ease of manipulation via php script dev-slp.php.
After conversion, the next step for modification is the necessity of having the input values between 0.0 and 1.0. For that, a function sva() is written in preparation.py. It converts the input letters to a number between 0.0-1.0 depending on encoding which we choose, just as we wanted.

Usage instructions

See codecopy.sh for some of the samples.

preparation.py has all the inputs and network parameters. This is the file where you would make changes to use the code for different usages.
Change inputwords and outputwords with your data. By default the database for samAsa classification is the input.
Set the various parameters of the network in preparation.py e.g. input_neuron, intermediate_neuron, epochs, mini_batch_size, eta, lmbda, howmany, mode.
Set whether you want to monitor the accuracies and costs in monitor_evaluation_cost, monitor_evaluation_accuracy, monitor_training_cost and monitor_training_accuracy. By default all of them are monitored.
Open your terminal.

Copy paste the code given below.

cd codes/SamaasaClassification
python -i
import preparation as p
training_data, test_data = p.load_data_wrapper(p.inputwords, p.outputwords, p.output_neuron, p.mode)
import network2
net = network2.Network([p.input_neuron, p.intermediate_neuron, p.output_neuron])
net.SGD(training_data, p.epochs, p.mini_batch_size, p.eta, p.all_class_types, p.mode, p.lmbda, evaluation_data=test_data, monitor_evaluation_accuracy=True, monitor_evaluation_cost=True, monitor_training_accuracy=True, monitor_training_cost=True, items=p.howmany)
net.save('networkstore.txt')

Explanations

cd codes/SamaasaClassification needs to be modified with your dictionary where the code is placed.
python -i opens the python terminal and ensures that the next code is executed in python shell.
import preparation as p imports the module preparation
training_data, test_data = p.load_data_wrapper(p.inputwords, p.outputwords, p.output_neuron, p.mode) loads the data from the inputwords, outputwords and majorwords to our network for use.
import network2 imports the network2.py which has the structure and code for self learning by ANN.
net = network2.Network([p.input_neuron, p.intermediate_neuron, p.output_neuron]) initialises our network named net.
net.SGD(training_data, p.epochs, p.mini_batch_size, p.eta, p.all_class_types, p.mode, p.lmbda, evaluation_data=test_data, monitor_evaluation_accuracy=True, monitor_evaluation_cost=True, monitor_training_accuracy=True, monitor_training_cost=True, items=p.howmany) - This is the core part of the code which actually learns from the input data. See network2.py for its parameters details. You may like / need to modify various parameters of this network to ensure that your ANN learns well.
net.save('networkstore.txt') saves the network details to networkstore.txt file, so that we don't have to repeat the whole training again and again.

Reloading a learnt ANN

cd codes/SamaasaClassification
python -i
import preparation
training_data, test_data = p.load_data_wrapper(p.inputwords,p.outputwords, p.output_neuron, p.mode)
import network2
net = network2.load('networkstore.txt')
net.SGD(training_data, p.epochs, p.mini_batch_size, p.eta, p.all_class_types, p.mode, p.lmbda, evaluation_data=test_data, monitor_evaluation_accuracy=True, monitor_evaluation_cost=True, monitor_training_accuracy=True, monitor_training_cost=True, items=p.howmany)
net.save('networkstore.txt')

The explanation remains the same.

Outputs

The console would show the accuracy and cost after every epoch.
The log file would store the following details: sizes, epochs, mini batch size, eta, lmbda, max training accuracy, max evaluation accuracy, min training cost, min evaluation cost, traininig accuracy, evaluation accuracy, weights, biases, unsolved examples, time of the training.
networkstore.txt file would store the network details for future reload.
wrongdata.txt file would store the strings which were wrongly classified by ANN.

Todo

Make it usable across UTF8 character range
Store the best accuracy network in all epochs, and not the last epoch network.
Store the wrong classification list for only the best accuracy epoch and not all epochs.

Results in samAsa classification

Classification in 5 major samAsa classes

Roughly 70 %.

[input_neuron, intermediate_neuron, output_neuron], epoch, mini_batch_size, eta, lmbda, howmany, mode : training_accuracy, evaluation_accuracy

[50, 50, 5], 100, 10, 1.5, 0.0, 1, "Sanskrit" : 71.9971616566, 70.0464396285

Classification in 55 minor samAsa classes

Roughly 55 %.