Tensorflow Implementation of Very Deep Convolutional Neural Network for Text Classification, proposed by Conneau et al.
Archiecture for VDCNN is now correctly re-implemented with Tensorflow 2 and tf.keras support. A simple training interface is implemented following Tensorflow 2 Expert Tutorial. Feel free to contribute additional utilities like TensorBoard support.
Side Note, if you are a newcomer for NLP text classification:
Please checkout new SOTA NLP methods like transformers or Bert.
Check out PyTorch for MUCH BETTER dynamic graphing and dataset object support.
The original paper tests several NLP datasets, including DBPedia, AG's News, Sogou News and etc.
tensorflow-datasets is used to support AG's News dataset.
Downloads of those NLP text classification datasets can be found here (Many thanks to ArdalanM):
Dataset | Classes | Train samples | Test samples | source |
---|---|---|---|---|
AG’s News | 4 | 120 000 | 7 600 | link |
Sogou News | 5 | 450 000 | 60 000 | link |
DBPedia | 14 | 560 000 | 70 000 | link |
Yelp Review Polarity | 2 | 560 000 | 38 000 | link |
Yelp Review Full | 5 | 650 000 | 50 000 | link |
Yahoo! Answers | 10 | 1 400 000 | 60 000 | link |
Amazon Review Full | 5 | 3 000 000 | 650 000 | link |
Amazon Review Polarity | 2 | 3 600 000 | 400 000 | link |
The original paper suggests the following details for training:
Some additional parameter settings for this repo:
Skip connections and pooling are correctly implemented now:
For dotted skip connections:
Please refer to Conneau et al for their methodology and experiment section in more detail.
Results are reported as follows: (i) / (ii)
TODO: Feel free to report your own experimental results in the following format:
Results for "Identity" Shortcut, "k-max" Pooling: | Depth | ag_news | DBPedia | Sogou News |
---|---|---|---|---|
9 layers | 90.17 / xx.xxxx | 98.44 / xx.xxxx | 96.42 / xx.xxxx | |
17 layers | 90.61 / xx.xxxx | 98.39 / xx.xxxx | 96.49 / xx.xxxx | |
29 layers | 91.33 / xx.xxxx | 98.59 / xx.xxxx | 96.82 / xx.xxxx | |
49 layers | xx.xx / xx.xxxx | xx.xx / xx.xxxx | xx.xx / xx.xxxx |
Original preprocessing codes and VDCNN Implementation By geduo15
Train Script and data iterator from Convolutional Neural Network for Text Classification