search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.15k stars 643 forks source link

It seems that the code does not just do softmax for the neighbor nodes, but just adds a bias_mat. I don't understand this step very well. #42

Open HelloCxz opened 4 years ago

HelloCxz commented 4 years ago

It seems that the code does not just do softmax for the neighbor nodes, but just adds a bias_mat. I don't understand this step very well.

PetarV- commented 4 years ago

Hello,

Thanks for your issue.

The bias matrix has entries of 0 in cells (i, j) where there exists an edge, and entries of -1e9 (approximation to negative infinity) when there does not exist an edge. When a softmax is applied to such a sum of logits and bias, this will effectively "mask out" non neighbouring nodes, as softmax of negative infinity is zero.

Hope that helps!

Thanks, Petar