This is a simple text classification library, based on keras. Some Arabic text normalization utilities are included.
1- Word Level CNN based on: "Convultion Neural Network for Text Classificartion" url: http://www.aclweb.org/anthology/D14-1181
2- Word Level C-LSTM based on: "A C-LSTM Neural Network for Text Classification" url:https://arxiv.org/pdf/1511.08630.pdf
3- Recurrent Network and its variants (BiLSTM, LSTM, GRU, BiGRU, Attention-BiLSTM)
4- Models implemented but currently not supported in options (Attention-LSTM,Attention-BiGRU).
5- Not yet tested (char level CNN).
Note: final model score is dumped into a file with name_of_model_score with both dev and test scores
This project utilize 6 deep learning models applied on Arabic Online Commentary Dataset
url: https://www.cs.jhu.edu/~ccb/publications/arabic-dialect-corpus.pdf
dataset url: https://www.cis.upenn.edu/~ccb/data/AOC-dialectal-annotations.zip
make sure to cite AOC oringial paper if you are going to use it in your work.
This work currently published in proceedings of VarDial Worshop 2018 co-located with COLING 2018 under the name "Deep Models for Arabic Dialect Identification on Benchmarked Data" [link]
Training data: [link]
Dev data: [link]
Test data: [link]
An example on how to use it is in: [link]
If you are going to follow up on this project please cite this work using the following bibtext:
@inproceedings{Elaraby2018,
title={Deep Models for Arabic Dialect Identification on Benchmarked Data},
author={Elaraby, Mohamed and Abdul-Mageed, Muhammad},
booktitle={Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial5)},
year={2018}
}
For Arabic Dialects we release 2 embedding models
AOC embedding: [Download URL]
Twitter Embedding Model: [Download URL]
@inproceedings{mageedYouTweet2018,
title={You Tweet What You Speak: A City-Level Dataset of Arabic Dialects},
author={Abdul-Mageed, Muhammad and Alhuzali, Hassan and Elaraby, Mohamed},
booktitle={LREC},
pages={3653--3659},
year={2018}
}