PyTorch implementation for news recommendation algorithms

For practice, I tried to implement the news recommendation algorithms in the recommenders with PyTorch

Implemented Algorithms

• NPA (Wu et al., 2019, SIGKDD) [1]
• NRMS (Wu et al., 2019, IJCNLP) [2]
• Fastformer (Wu et al., 2021, arxiv) [3]

Dataset

Take it easy, do not download the dataset first, We will automatically download the dataset in train.py

• MIND: the description about MIND dataset can refer to this document

Environment

• OS: Ubuntu 18.04
• Python == 3.7.13
• mlflow == 1.29.0

Please follow the setup steps bellow:

1. Install Anaconda
2. Create a virtual environment by using conda command

   $> conda env create -n deepnewsrec python=3.7.13

3. Activate the virtual environment we created

   $> conda activate deepnewsrec

4. Install mlflow

   (deepnewsrec)$> pip install mlflow==1.29.0

How to run the code

• Try a toy example with demo dataset

  $> mlflow run -e train --experiment-name individual_runs -P mind_type=demo -P epochs=1 .

  or you can use git uri

  $> mlflow run -e train --experiment-name individual_runs -P mind_type=demo -P epochs=1 git@github.com:hongjunyan/pytorch-news-recommendation.git 

• Run hyperparameter tuning

  $> mlflow run -e tune_with_ray --experiment-name tune_hyperparams -P mind_type=demo -P epochs=1 .

  For training a formal model, please use mind_type=large.

MLFlow UI

The experiment results recorded in /mlruns. You can execute mlflow ui to see the results.

$> mlflow ui -h 127.0.0.1

then hit http://127.0.0.1:5000

Results of Formal Model

I train the bellowing models with epochs=15, batch_size=128

Related projects

• recommenders: best practices for building recommendation systems

Reference papers

• [1] Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019b. Npa: Neural news recommendation with personalized attention. In KDD, pages 2576–2584. ACM.
• [2] Chuhan Wu, Fangzhao Wu, Suyu Ge, Tao Qi, Yongfeng Huang, and Xing Xie. 2019c. Neural news recommendation with multi-head selfattention. In EMNLP-IJCNLP, pages 6390–6395.
• [3] Chuhan Wu, Fangzhao Wu, Tao Qi, Yongfeng Huang, and Xing Xie. 2021. Fastformer: Additive attention can be all you need. arXiv preprint arXiv:2108.09084(2021)

TroubleShooting:

• Nan in Tensor (refer to FastSelfAttention in models/fastformer.py): https://discuss.pytorch.org/t/why-i-keep-getting-nan/20383