search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.18k stars 642 forks source link

What is the difference between GCN and Const-GAT? #9

Closed wzm2256 closed 6 years ago

wzm2256 commented 6 years ago

Const-GAT is describe as "Const-GAT corresponds to a model with the same architecture as GAT, but with a constant attention mechanism (assigning same importance to each neighbor; GCN-like inductive operator) ". In my opinion, that is exactly what GCN is.

Could you explain a little bit?

PetarV- commented 6 years ago

Const-GAT is not quite GCN---the GCN will assign the weight of 1/sqrt(deg(u)deg(v)) for edge u <-> v. Const-GAT will, instead, effectively assign 1/deg(u) for edge u <- v, and 1/deg(v) for edge u -> v.

Critically, we wanted an inductive operator---one that will not be aware of the degrees of neighbours; as in the inductive setting, (test) nodes can be incrementally added into the graph, and hence the degrees may dynamically change.

(Keep in mind that Const-GAT is, technically, not even aware of deg(u)---it just receives scores of 1 everywhere, which it softmax-normalises to effectively get 1/deg(u) coefficients.)

The purpose of the Const-GAT experiment was, as suggested by one of our reviewers, to directly assess the impact of our attention mechanism, as opposed to our other architectural decisions.

wzm2256 commented 6 years ago

OK, it's true. I now agree Const_GAT is different from GCN. But now I'm wondering why the coefficient stuff could make such a huge difference.

Thanks for your answer and patience.