Neural Essay Assessor

An automatic essay scoring system based on convolutional and recurrent neural networks, including GRU and LSTM.

Set Up

• Install Keras (with Theano backend)
• Prepare data
• Run train_nea.py

Packages

• nltk

  pip install ntlk

  import nltk
  nltk.download('punkt')

Data

We have used 5-fold cross validation on ASAP dataset to evaluate our system. This dataset (training_set_rel3.tsv) can be downloaded from here. After downloading the file, put it in the data directory and create training, development and test data using preprocess_asap.py script:

cd data
python preprocess_asap.py -i training_set_rel3.tsv

Options

You can see the list of available options by running:

python train_nea.py -h

Example

The following command trains a model for prompt 1 in the ASAP dataset, using the training and development data from fold 0 and evaluates it.

THEANO_FLAGS="device=gpu0,floatX=float32" python train_nea.py
    -tr data/fold_0/train.tsv
    -tu data/fold_0/dev.tsv
    -ts data/fold_0/test.tsv
    -p 1    # Prompt ID
    --emb embeddings.w2v.txt
    -o output_dir

Frequently Asked Questions

See our FAQ page for a list of frequently asked questions. If the answer to your question is not there, contact me (kaveh@comp.nus.edu.sg).

License

Neural Essay Assessor is licensed under the GNU General Public License Version 3. Separate commercial licensing is also available. For more information contact:

• Kaveh Taghipour (kaveh@comp.nus.edu.sg)
• Hwee Tou Ng (nght@comp.nus.edu.sg)

Publication

Kaveh Taghipour and Hwee Tou Ng. 2016. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.