text classification

[nnop]

researcher

• Rie Johnson
  • CONTEXT v4: Neural network code for text categorization in C++ on GPU

papers

• Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan. "Thumbs up?: sentiment classification using machine learning techniques." EMNLP (2002).
• Zhang, Ye and Byron C. Wallace. “A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification.” CoRR abs/1510.03820 (2015): n. pag.
• Yang, Zichao et al. “Hierarchical Attention Networks for Document Classification.” HLT-NAACL (2016).
• Johnson, Rie and Tong Zhang. “Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level.” CoRR abs/1609.00718 (2016): n. pag.
• Johnson, Rie and Tong Zhang. “Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level.” CoRR abs/1609.00718 (2016): n. pag.
• Johnson, Rie and Tong Zhang. “Deep Pyramid Convolutional Neural Networks for Text Categorization.” ACL (2017).

[nnop]

可视化

• https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/5_word2vec.ipynb

[nnop]

网络深度

• NLP best practice - Depth

In most cases, however, performance improvements of making the model deeper than 2 layers are minimal (Reimers & Gurevych, 2017). These observations hold for most sequence tagging and structured prediction problems. For classification, deep or very deep models perform well only with character-level input and shallow word-level models are still the state-of-the-art (Zhang et al., 2015; Conneau et al., 2016; Le et al., 2017)

[nnop]

优化

• NLP best practice - Optimization

Adam (Kingma & Ba, 2015) is one of the most popular and widely used optimization algorithms and often the go-to optimizer for NLP researchers. It is often thought that Adam clearly outperforms vanilla stochastic gradient descent (SGD). However, while it converges much faster than SGD, it has been observed that SGD with learning rate annealing slightly outperforms Adam (Wu et al., 2016). Recent work furthermore shows that SGD with properly tuned momentum outperforms Adam (Zhang et al., 2017)

[nnop]

预处理

• 用u'([\u4E00-\u9FA5a-zA-Z0-9+_]+)'去掉特殊字符和标点（注意因为是unicode范围，输入word需要decode(‘utf8’)）；

[nnop]

text classification

• [EMNLP 2014] Convolutional Neural Networks for Sentence Classification
  • codes
  • slides
• [AAAI 2016] Character-Aware Neural Language Models
• Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks
• Recurrent Convolutional Neural Networks for Text Classification
• How to do text classification with CNNs, TensorFlow and word embedding

codes

• https://github.com/brightmart/text_classification
• https://github.com/Shawn1993/cnn-text-classification-pytorch

[nnop]

知乎·看山杯

• 「2017 知乎 · 看山杯机器学习挑战赛」结束，谁获奖了？知乎还会做什么？
• 2017知乎看山杯总结(多标签文本分类)

[nnop]

发现了一个新思路

• semi-supervised/unsupervised corpora construction