search

graphdeeplearning / graphtransformer

Graph Transformer Architecture. Source code for "A Generalization of Transformer Networks to Graphs", DLG-AAAI'21.
https://arxiv.org/abs/2012.09699
MIT License
889 stars 137 forks source link

Eval sign flipping #22

Closed devnkong closed 2 years ago

devnkong commented 2 years ago

Hi Vijay,

Thanks for your repo!

Question: I see your doing sign flipping of eigen pos_enc during training, but it seems that you are not doing so during eval time. I understand that we want to make deterministic predictions so we don't have random flipping when evaluating it. Do you have further comments or justification for this?

Best Kezhi

devnkong commented 2 years ago

Also do you have some reason for choosing eigen vectors with small eigen values?

vijaydwivedi75 commented 2 years ago

Hi @devnkong, thanks for your questions.

Q: Why sign flipping is not used during eval? A: The random sign flipping during the training is to allow the network to be invariant or independent of the choices among 2^k possibilities. By this approach then, the sign flipping is not required during eval.

Q: choosing eigen vectors with small eigen values? A: Please refer to Section E.1.2 in https://arxiv.org/pdf/2003.00982.pdf

Best, Vijay

devnkong commented 2 years ago

Thanks!