search

microsoft / Graphormer

Graphormer is a general-purpose deep learning backbone for molecular modeling.
MIT License
2.08k stars 334 forks source link

Edge Understanding of Graphormer #31

Closed egeozsoy closed 2 years ago

egeozsoy commented 2 years ago

It looks like the only use of edge encodings right now is to change the bias of the attention, so in a toy example where there are just two identical nodes with a single edge between them, with the edge label(edge feature) either 0 or 1, and a binary classification task, where the wanted prediction is the label of this edge, how would the network solve this task? Just using the edge encoding as attention bias should not be enough here right? I am asking because, I successfully applied graphormer to a similer task, but now I am not exactly sure how it works

zhengsx commented 2 years ago

Hi @egeozsoy , glad to see that Graphormer works on your task. Indeed, for the case about two identical nodes, edge encoding could not help it to distingush each other due to the weighted-sum (in self-attention) would fail when the representations of node are same. The possible reason of why Graphormer is successfully applied to the similar task, might be that the two nodes are not exactly same or there is a third node in your actual system.

zhengsx commented 2 years ago

Close this issue due to inactive. Feel free to raise a new one or reopen this one for any further question.