Text-Classification-Models-Pytorch

Implementation of State-of-the-art Text Classification Models in Pytorch

Implemented Models

• fastText: fastText Model from Bag of Tricks for Efficient Text Classification
• TextCNN: CNN for text classification proposed in Convolutional Neural Networks for Sentence Classification
• TextRNN: Bi-direction LSTM network for text classification
• RCNN: Implementation of RCNN Model proposed in Recurrent Convolutional Neural Networks for Text Classification
• CharCNN: Implementation of character-level CNNs as proposed in the paper Character-level Convolutional Networks for Text Classification
• Seq2seq With Attention: Implementation of seq2seq model with attention from Neural Machine Translation by Jointly Learning to Align and Translate, Text Classification Research with Attention-based Recurrent Neural Networks
• Transformer: Implementation of Transformer model proposed in Attention Is All You Need

Requirements

• Python-3.5.0
• Pandas-0.23.4
• Numpy-1.15.2
• Spacy-2.0.13
• Pytorch-0.4.1.post2
• Torchtext-0.3.1

Usage

1) Download data into "data/" directory or use already available data 2) If using your own data, convert it into the same format as of provided data 3) Download Pre-trained word embeddings (Glove/Word2Vec) into "data/" directory 4) Go to corresponding model directory 5) run following command:

python train.py

Model Performance

• All the models were run on a 14GB machine with 2 Cores and one NVIDIA Tesla K80 GPU.
• Runtime in the table below includes the time to load and process the data and running the model.
• Model parameters are not tuned. So, better performance can be achieved by some parameter tuning.

References

[1] Bag of Tricks for Efficient Text Classification
[2] Convolutional Neural Networks for Sentence Classification
[3] Recurrent Convolutional Neural Networks for Text Classification
[4] Character-level Convolutional Networks for Text Classification
[5] Neural Machine Translation by Jointly Learning to Align and Translate
[6] Text Classification Research with Attention-based Recurrent Neural Networks
[7] Attention Is All You Need
[8] Rethinking the Inception Architecture for Computer Vision
[9] Identifying Well-formed Natural Language Questions