2021.IJCAI.User as Graph User Modeling with Heterogeneous Graph Pooling

[soroush-ziaeinejad]

Main problem:

This paper introduces a method for user modeling considering the dependency between user behaviors that are obtained from users’ clicks on the news articles and calls it User-as-Graph (UaG). It also represents a method for heterogeneous graph pooling using graph neural networks to learn user interest embedding. The main goal is to predict the clicks on news articles.

Existing shortcomings:

Existing methods for news recommendation are:

• Ignoring the important relatedness between user behaviors.
• Modeling users’ interest just via sequential or attentive models from the clicks information.

SOTA:

• GNN to embed user-news-topic graph / Hu.IP&M.2020
• Using the neighbors of a node to embed a user-news graph / Gu.WWW.2020

The improvement: in the above-mentioned methods, each user is a node in the user-news graph. Proposed method models each user to a personalized graph using users’ behaviors (each user is a graph).

UaG:

• Each user is a heterogeneous graph
• Each graph has three types of nodes: news, topic, entity
• Three types of relatedness between user behaviors:
  • Users click on temporally adjacent news articles.
  • Users click on news articles with the same topics.
  • Users click on news articles that contain common entities (keywords)

User interest embedding:

• Approach: Graph pooling. Problem: existing methods are for homogeneous graphs, but their graphs have three types of nodes with different characteristics and structures. Solution: proposing a heterogeneous graph pooling (HG-Pool)
• Goal: to condense the heterogeneous graph into a small dense graph and then apply the ‘DiffPool’ layer to achieve a single node containing all of the main graph information.

News embedding:

• Embedding news using news encoder
• Inputs of news encoder are the news titles, topics, and entities (keywords)
• Output of the news encoder is the embedding of each news article using an attention network

News scores:

Finally, the score of each news article (click probability) is computed by the inner product of user embedding and news embedding outputs.

Experiments:

• Dataset: MIND (1M user, 6 weeks in 2019, Microsoft News)
• GNN: GAT
• Word embedding: Glove (2014)
• User embedding layers: 2 HG-Pool, 1 DiffPool
• Optimizer: Adam
• Metrics: [AUC, MRR, NDCG@5, NDCG@10]

Baselines (NM is used as News Recommendation):

• LibFM: Factorization Machines for NM (FM) / TIST12
• EBNR: autoencoders and gated recurrent networks for NM (using LDA) / KDD17
• DKN: Deep Knowledge-aware networks for NM / WWW18
• NPA: NM via personalized attention / KDD19
• NAML: Attentive multi-view learning for NM / IJCAI19
• LSTUR: Long-term-short-term user representations for NM / IJCAI19
• NRMS: Multi-head self-attention / EMNLP-IJCNLP19
• GNewsRec: LSTM and GNN for modeling short-term and long-term user interests respectively / IP&M20
• GERL: using neighbors of news and users in user-news graph / WWW20

Code:

Unfortunately, there is no available implementation for this paper.

[soroush-ziaeinejad]

@hosseinfani, Please take a look at the google scholar page of the first author. How a PhD student can submit 27 papers in one year?! https://scholar.google.com/citations?hl=en&user=OG1cMswAAAAJ&view_op=list_works&citft=1&email_for_op=soroushziaeinejad%40gmail.com&sortby=pubdate

[hosseinfani]

@soroush-ziaeinejad thank you. I really like your summaries. please look at his homepage. you'll be more surprised :)

[soroush-ziaeinejad]

Thanks Hossein. Yes I saw that too!! It's not normal at all :D