search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.18k stars 642 forks source link

About structural information #5

Closed 1049451037 closed 6 years ago

1049451037 commented 6 years ago

Hello Petar, I am very interested in your recent work on GAT and it inspires me a lot. However, I have a question about structural information of graph. When and where to feed structure information of graph into GAT? I can only see the bias matrix restriction in the code. Is it the only structural information fed into GAT? Looking forward to your reply~

PetarV- commented 6 years ago

Hi Qingsong,

Thank you for your note, kind words and your interest in the GAT paper!

Yes, we currently only feed in the graph structure into GATs by restricting the self-attention mechanism to act only along the edges of the graph. This enriches the model with inductive properties; i.e. allows us to apply it to arbitrary structures at test time.

If you think you will not require such properties, e.g. if you'd like the model to behave more like GCN or the presented implementations of MoNet, you can attach 1/sqrt(degree) to each node as an additional feature. Then the attention mechanism will be able to directly exploit structural information when computing coefficients.

Thanks, Petar