search

graphdeeplearning / graphtransformer

Graph Transformer Architecture. Source code for "A Generalization of Transformer Networks to Graphs", DLG-AAAI'21.
https://arxiv.org/abs/2012.09699
MIT License
889 stars 137 forks source link

about attention #5

Closed xinsheng44 closed 3 years ago

xinsheng44 commented 3 years ago

图片 图片 Hello, about the problem of calculating attention, the attention of node i and its adjacent nodes is calculated in the formula, but I find that the final calculation is the attention of all nodes, and it does not distinguish whether the nodes are connected. Is there a problem with my understanding?

vijaydwivedi75 commented 3 years ago

Hi @lao-sheng,

The code section you highlight give the outputs of Q, K, V projections.

The implementation for node to attend to its local neighborhood is carried out by this function, which is just followed after the projections. https://github.com/graphdeeplearning/graphtransformer/blob/3c83b4ba5e45a2e25bbefde1b35d88a27ca3cfb2/layers/graph_transformer_layer.py#L73

https://github.com/graphdeeplearning/graphtransformer/blob/3c83b4ba5e45a2e25bbefde1b35d88a27ca3cfb2/layers/graph_transformer_layer.py#L51-L59

xinsheng44 commented 3 years ago

oh,i see.thanks！