search

leffff / graphormer-pyg

Microsoft Graphormer (https://arxiv.org/abs/2106.05234) rewritten in Pytorch-Geometric
MIT License
125 stars 17 forks source link

The code will be very slow. #24

Open StefanIsSmart opened 6 months ago

StefanIsSmart commented 6 months ago

You use the for-loop for multi-head. (Time x The Number of Heads) And also use the for-loop for Graph attention. (Time x The Number of Graph)

It will be very slow.

Is there any other way to solve that point ?

leffff commented 6 months ago

Not sure yet! I believe the best way is to 1) Rewrite it to classic mutlihead attentinon 2) Rewrite "1)" to PyG-like attention