search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.15k stars 643 forks source link

Weighted adjacency graphs #63

Closed JoakimHaurum closed 3 years ago

JoakimHaurum commented 3 years ago

Hi,

I am currently working on a project where we have weighted directed graphs. I know that the current model support directed graphs, i.e. the neighborhood is defined as the incoming nodes.

However, is it possible to use weighted adjacency graphs? My intuition is saying no, as the edge weights are learned through the attention mechanisms. However, in the associated blog there is a footnote that states that GAT can be trivially extended to include these cases ( https://petar-v.com/GAT/#fn:1 ).

So if it is possible, how would one do this?

PetarV- commented 3 years ago

Hi Joakim,

Thank you for your issue. The easiest/most intuitive way to include the weights inside an attentional model is to feed the weights as an additional input to the attention mechanism. One easy way of doing so is adding or multiplying (potentially affine-transformed) weights to the attention coefficients prior to applying softmax.

Hope that helps!

Thanks, Petar

JoakimHaurum commented 3 years ago

Thank you for the quick response Petar!

It makes sense, so basically just multiply e_ij with a_ij, where a_ij is the weight of the edge from node j to node i (in my case in the range [0, 1]), I like the idea of simply multiplying the raw edge weight as it represents an a priori scaling of the connection and generalizes to the standard GAT when a_ij = 1, whereas addition does not have an as nice interpretation.