search

pyg-team / pytorch_geometric

Graph Neural Network Library for PyTorch
https://pyg.org
MIT License
20.95k stars 3.61k forks source link

Which graph convolutional layers can be trained on multiple graphs? #1149

Closed douglasrizzo closed 4 years ago

douglasrizzo commented 4 years ago

❓ Questions & Help

Hi, this is more of a theoretical question. The GAT paper mentions that there are some graph convolutional layers that depend on structural information of the graph they will be applied on, which makes it impossible to train them on multiple graphs or apply them on a different graph than the one it was originally trained on. For example, GCN and its predecessors depend on the Laplacian matrix or the adjacency matrix of a graph, which seems to tie it to a single graph.

The GAT paper lists itself and GraphSAGE as models that may be applied to multiple graphs. While I go through each of the layer types looking for answers, I might as well ask: are there other convolutional layers in PyG that are detached from the structure of the graph and can be applied to multiple graphs?

rusty1s commented 4 years ago

While it is true that ChebConv and GCNConv are motivated by the Eigenbasis of the Laplacian matrix (and are hence restricted to operating on single graphs), their simplifications/modifications allow them to be applied to multiple graphs as well (since they all work in a spatial localized fashion). That is, all the GNN operators in PyG are generally suitable for operating on multiple graphs.

bknyaz commented 4 years ago

I agree with @rusty1s, just let me share three papers that analyze this question theoretically and empirically supporting that most spectral GCNs work just fine on multiple graphs:

  1. Bridging the Gap Between Spectral and Spatial Domains in Graph Neural Networks, https://arxiv.org/abs/2003.11702
  2. On the transferability of spectral graph filters, https://arxiv.org/abs/1901.10524
  3. Spectral multigraph networks for discovering and fusing relationships in molecules, https://arxiv.org/abs/1811.09595
douglasrizzo commented 4 years ago

Thanks @rusty1s. I saw that DGL had a different equation for the GCN operator, more in line with a spatial representation of the convolution and not a spectral one. I just wasn't sure with regards to PyG, as its definition of GCNConv is the one from the paper, which comes from the spectral convolution. Also thanks @bknyaz, I'll definitely check the linked papers.

douglasrizzo commented 4 years ago

Just to add a little more info, I was pointed to the paper on Message Passing Neural Networks, which has an interpretation of Laplacian-based models as MPNNs in the appendix.

Neural Message Passing for Quantum Chemistry: https://arxiv.org/abs/1704.01212