search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.13k stars 642 forks source link

Question about GAT attention training #52

Open l-hoang opened 3 years ago

l-hoang commented 3 years ago

Hello.

To my understanding, the self attention step uses a learnable weight vector and a neural network. However, the code in this repo seems to use a very simple constant self-attention step.

https://github.com/PetarV-/GAT/blob/77d0d41b432df79184beb64bb6fcab557851e0b4/utils/layers.py#L13

Is my understanding incorrect? Is there a place in the code where the neural network used for the self-attention step is trained?

Thank you.

PetarV- commented 3 years ago

Hi,

Thank you for your issue! The neural network consists of the two conv1d layers applied right before the attention (these layers contain the weight vector). Backpropagating will hence train these networks along with every other part of the GAT.

Hope that helps!

Thanks, Petar