search

acbull / pyHGT

Code for "Heterogeneous Graph Transformer" (WWW'20), which is based on pytorch_geometric
MIT License
775 stars 162 forks source link

Question on weighted graph #33

Open xianshu1 opened 3 years ago

xianshu1 commented 3 years ago

I have a weighted graph, where each edge has weight assign to it. Is it possible to incorporate edge weight into the learning as well? If yes, could you advise how shall I do this? Thank you.

acbull commented 3 years ago

Hi.

Our model indeed supports encode edge information. In our case, we treat discrete timestamp as the edge feature and adopt a temporal encoding (https://github.com/acbull/pyHGT/blob/16547c0c0a6977c40b8efa88a7f3e40cf1955362/pyHGT/conv.py#L283) to encode it, which will be added to the source node when passing to the target.

If your feature is continuous (like weight), you can simply adopt another encoding method to project them into a d-dimensional vector, and replace the sinusoid embedding table in the RTE, and then the weights (edge feature) could be encoded.

xianshu1 commented 3 years ago

Thank you for your quick reply. This helps a lot. Do you have suggestions on what encoding function would work well for continuous feature (like weight) to project them into a d-dimensional vector?

acbull commented 3 years ago

You can try this paper: http://proceedings.mlr.press/v119/liu20n/liu20n.pdf

xianshu1 commented 3 years ago

Thank you for the reference.

I am wondering if an alternative way to incorporate edge weight would be multiply edges weights to the heterogeneous mutual attention coefficients prior to apply softmax?

acbull commented 3 years ago

Let's doable, but I guess that's not the optimal solution, as the model should learn the attention by itself.

If you wants to do that, simply reply the temporal stamp as weght vector, then product it before softmax

On Thu, Feb 4, 2021, 23:06 xianshu1 notifications@github.com wrote:

Thank you for the reference.

I am wondering if an alternative way to incorporate edge weight would be multiply edges weights to the heterogeneous mutual attention coefficients prior to apply softmax?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/acbull/pyHGT/issues/33#issuecomment-773772639, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHREXR45Q5RU5YCGNFI5XHLS5NVDNANCNFSM4XCEIFWA .